Alignment of master and slave in a minimally invasive surgical apparatus

ABSTRACT

This invention relates to establishing alignment or a desired orientational relationship between a master and a slave of a telerobotic system. The invention can advantageously be used in a surgical apparatus. A method of establishing a desired orientational relationship between a hand-held part of a master control and an end effector of an associated slave as viewed in an image displayed on a viewer is provided. The method includes causing the end effector to remain stationary, determining a current orientation of the end effector relative to a viewing end of an image capturing device operatively associated with the viewer and determining a desired corresponding orientation of the hand-held part of the master control relative to the viewer, at which orientation the desired orientational relationship between the hand-held part of the master control and the end effector would be established. The method further includes causing the hand-held part of the master control to be moved into the desired corresponding orientation. The invention extends to a control system arranged to cause the desired orientational relationship between the hand-held part of the master control and the end effector of the associated slave, as viewed in the displayed imaged on the viewer, to be established when operative control between the master control and the slave has been interrupted.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is in continuation and related to the following patentsand patent applications, the fill disclosures of which are incorporatedherein by reference: PCT International Application No. PCT/US98/19508,entitled “Robotic Apparatus,” filed Sep. 18, 1998, U.S. application Ser.No. 60/111,710, entitled “Master Having Redundant Degrees of Freedom,”filed on Dec. 8, 1998, U.S. application Ser. No. 60/111,713, entitled“Surgical Robotic Tools, Data Architecture, and Use,” filed on Dec. 8,1998; U.S. application Ser. No. 60/111,711, entitled “Image-Shifting fora Telerobotic System,” filed on Dec. 8, 1998; U.S. application Ser. No.60/111,714, entitled “Stereo Viewer System for Use in TeleroboticSystems,” filed on Dec. 8, 1998; U.S. application Ser. No. 60/116,842,entitled “Repositioning and Reorientation of Master/Slave Relationshipin Minimally Invasive Telesurgery,” filed on Jan. 22, 1999; and Ser. No.08/709,965, filed Sep. 9, 1996, now U.S. Pat. No. 5,808,665, entitled“Endoscopic Surgical Instrument and Method for Use,” issued on Sep. 15,1998; the full disclosures of which are incorporated herein byreference.

BACKGROUND OF THE INVENTION

Minimally invasive medical techniques are aimed at reducing the amountof extraneous tissue which is damaged during diagnostic or surgicalprocedures, thereby reducing patient recovery time, discomfort, anddeleterious side effects. Millions of surgeries are performed each yearin the United States. Many of these surgeries can potentially beperformed in a minimally invasive manner. However, only a relativelysmall number of surgeries currently use these techniques due tolimitations in minimally invasive surgical instruments and techniquesand the additional surgical training required to master them.

Advances in minimally invasive surgical technology could dramaticallyincrease the number of surgeries performed in a minimally invasivemanner. The average length of a hospital stay for a standard surgery issignificantly longer than the average length for the equivalent surgeryperformed in a minimally invasive surgical manner. Thus, the completeadoption of minimally invasive techniques could save millions ofhospital days, and consequently millions of dollars annually in hospitalresidency costs alone. Patient recovery times, patient discomfort,surgical side effects, and time away from work are also reduced withminimally invasive surgery.

The most common form of minimally invasive surgery is endoscopy.Probably the most common form of endoscopy is laparoscopy, which isminimally invasive inspection and surgery inside the abdominal cavity.In standard laparoscopic surgery, a patient's abdomen is insufflatedwith gas, and cannula sleeves are passed through small (approximately ½inch) incisions to provide entry ports for laparoscopic surgicalinstruments.

The laparoscopic surgical instruments generally include a laparoscopefor viewing the surgical field, and working tools defining endeffectors. Typical surgical end effectors include clamps, graspers,scissors, staplers, or needle holders, for example. The working toolsare similar to those used in conventional (open) surgery, except thatthe working end or end effector of each tool is separated from itshandle by, e.g., an approximately 12-inch long, extension tube.

To perform surgical procedures, the surgeon passes these working toolsor instruments through the cannula sleeves to a required internalsurgical site and manipulates them from outside the abdomen by slidingthem in and out through the cannula sleeves, rotating them in thecannula sleeves, levering (i.e., pivoting) the instruments against theabdominal wall and actuating end effectors on the distal ends of theinstruments from outside the abdomen. The instruments pivot aroundcenters defined by the incisions which extend through muscles of theabdominal wall. The surgeon monitors the procedure by means of atelevision monitor which displays an image of the surgical site via alaparoscopic camera. The laparoscopic camera is also introduced throughthe abdominal wall and into the surgical site. Similar endoscopictechniques are employed in, e.g., arthroscopy, retroperitoneoscopy,pelviscopy, nephroscopy, cystoscopy, cisternoscopy, sinoscopy,hysteroscopy, urethroscopy and the like.

There are many disadvantages relating to current minimally invasivesurgical (MIS) technology. For example, existing MIS instruments denythe surgeon the flexibility of tool placement found in open surgery.Most current laparoscopic tools have rigid shafts and difficulty isexperienced in approaching the surgical site through the small incision.Additionally, the length and construction of many surgical instrumentsreduces the surgeon's ability to feel forces exerted by tissues andorgans on the end effector of the associated tool. The lack of dexterityand sensitivity of surgical tools is a major impediment to the expansionof minimally invasive surgery.

Minimally invasive telesurgical systems for use in surgery are beingdeveloped to increase a surgeon's dexterity as well as to allow asurgeon to operate on a patient from a remote location. Telesurgery is ageneral term for surgical systems where the surgeon uses some form ofremote control, e.g., a servomechanism, or the like, to manipulatesurgical instrument movements rather than directly holding and movingthe instruments by hand. In such a telesurgery system, the surgeon isprovided with an image of the surgical site at the remote location.While viewing typically a three-dimensional image of the surgical siteon a suitable viewer or display, the surgeon performs the surgicalprocedures on the patient by manipulating master control devices, at theremote location, which control the motion of servomechanically operatedinstruments.

The servomechanism used for telesurgery will often accept input from twomaster controllers (one for each of the surgeon's hands), and mayinclude two robotic arms. Operative communication between each mastercontrol and an associated arm and instrument assembly is achievedthrough a control system. The control system includes at least oneprocessor which relays input commands from a master controller to anassociated arm and instrument assembly and from the arm and instrumentassembly to the associated master controller in the case of, e.g., forcefeedback.

It would be advantageous if the control system establishes arelationship between master control and associated end effector asviewed in the image, in which relationship the orientation and positionof the end effector as displayed to the surgeon in the image follows andcorresponds with the orientation and position of the surgeon's hand whenmanipulating the associated master controls. Thus, it would beadvantageous if end effector orientation and position is mapped ontosurgeon hand orientation and position. In such a case control of endeffector movement would be performed in a more intuitive manner than inthe case where end effector movement as displayed in the image were notmapped onto surgeon hand movement.

It would further be advantageous if the position of the image capturingdevice could be changed during the course of a surgical procedure so asto enable the surgeon to view the surgical site from another position.It will be appreciated that, should the image capturing device positionchange, the orientation and position of the end effectors in the viewedimage could also change. It would further be advantageous if therelationship in which end effector movement is mapped onto hand movementcould again be established after such an image capturing devicepositional change.

It is an object of the invention to provide a method and control systemwhich permits the mapping of end effector movement onto hand movement tobe established and/or reestablished after having been interrupted, forexample, by an image capturing device positional change.

It is to be appreciated that although the method and control system ofthe invention is described with reference to a minimally invasivesurgical apparatus in this specification, the application of theinvention is not to be limited to this application only, but can be usedin any type of apparatus where an input is entered at one location and acorresponding movement is required at a remote location and in which itis required, or merely beneficial, to realign or re-map slaveorientational and/or positional movement onto master input movementsafter interruption.

SUMMARY OF THE INVENTION

In accordance with one aspect of the invention, there is provided amethod of establishing or reestablishing a desired orientationalrelationship between a hand-held part of a master control and an endeffector of an associated slave as viewed in an image displayed on aviewer, the method including causing the end effector to remainstationary; determining a current orientation of the end effectorrelative to a viewing end of an image capturing device operativelyassociated with the viewer; determining a desired correspondingorientation of the hand-held part of the master control relative to theviewer at which orientation the desired orientational relationshipbetween the hand-held part of the master control and the end effectorwould be established or reestablished; and causing the hand-held part ofthe master control to be moved into the desired correspondingorientation.

The method may further include determining a current position of the endeffector relative to the viewing end of the image capturing deviceoperatively associated with the viewer; and determining a desiredcorresponding position of the hand-held part of the master controlrelative to the viewer at which position a desired positionalrelationship between the hand-held part of the master control and theend effector would be established or reestablished; and causing thehand-held part of the master control to be moved into the desiredcorresponding position.

According to another aspect of the invention, there is provided asurgical control system arranged to cause a desired orientationalrelationship between a hand-held part of a master control and an endeffector of an associated slave, as viewed in a displayed image on aviewer, to be established or reestablished when operative controlbetween the master control and the slave has been interrupted, thecontrol system including processing means arranged to cause the endeffector to remain stationary; processing means arranged to compute acurrent orientation of the end effector relative to a viewing end of animage capturing device operatively associated with the viewer; and tocompute a desired corresponding orientation of the hand-held part of themaster control at which orientation the desired orientationalrelationship between the hand-held part of the master control and theend effector would be established or reestablished; and an actuatorarranged to cause the hand-held part of the master control to be movedinto the desired orientation.

The surgical control system may further include processing meansarranged to compute a current position of the end effector relative tothe viewing end of the image capturing device operatively associatedwith the viewer; and to compute a desired corresponding position of thehand-held part of the master control at which position the desiredpositional relationship between the hand-held part of the master controland the end effector would be established or reestablished; and anactuator arranged to cause the hand-held part of the master control tobe moved into the desired position.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention will now be described, by way of example, and withreference to the accompanying diagrammatic drawings, in which:

FIG. 1A shows a three-dimensional view of an operator station of atelesurgical system in accordance with the invention;

FIG. 1B shows a three-dimensional view of a cart or surgical station ofthe telesurgical system, the cart carrying three robotically controlledarms, the movement of the arms being remotely controllable from theoperator station shown in FIG. 1A;

FIG. 2A shows a side view of a robotic arm and surgical instrumentassembly;

FIG. 2B shows a three-dimensional view corresponding to FIG. 2A;

FIG. 3 shows a three-dimensional view of a surgical instrument;

FIG. 4 shows a schematic kinematic diagram corresponding to the sideview of the robotic arm shown in FIG. 2A, and indicates the arm havingbeen displaced from one position into another position;

FIG. 5 shows, at an enlarged scale, a wrist member and end effector ofthe surgical instrument shown in FIG. 3, the wrist member and endeffector being movably mounted on a working end of a shaft of thesurgical instrument;

FIG. 6A shows a three-dimensional view of a hand held part or wristgimbal of a master control device of the telesurgical system;

FIG. 6B shows a three-dimensional view of an articulated arm portion ofthe master control device of the telesurgical system on which the wristgimbal of FIG. 6A is mounted in use;

FIG. 6C shows a three-dimensional view of the master control deviceshowing the wrist gimbal of FIG. 6A mounted on the articulated armportion of FIG. 6B;

FIG. 7 shows a schematic three-dimensional drawing indicating thepositions of the end effectors relative to a viewing end of an endoscopeand the corresponding positions of master control devices relative tothe eyes of an operator, typically a surgeon;

FIG. 8 shows a schematic three-dimensional drawing indicating theposition and orientation of an end effector relative to a cameraCartesian coordinate reference system;

FIG. 9 shows a schematic three-dimensional drawing indicating theposition and orientation of a pincher formation of the master controldevice relative to an eye Cartesian coordinate reference system;

FIG. 10 shows a schematic side view of part of the surgical station ofthe minimally invasive surgical apparatus indicating the location ofCartesian reference coordinate systems used by a control system of theminimally invasive surgical apparatus to determine the position andorientation of an end effector relative to a Cartesian referencecoordinate system at the viewing end of an image capturing device;

FIG. 11 shows a schematic side view of part of the operator station ofthe minimally invasive surgical apparatus indicating the location ofCartesian reference coordinate systems used by the control system of theminimally invasive surgical apparatus to determine the position andorientation of the pincher formation of the master control devicerelative to an eye Cartesian reference coordinate system;

FIG. 12 shows a schematic block diagram indicating steps followed by thecontrol system of the minimally invasive surgical apparatus indetermining end effector position and orientation relative to theCartesian reference coordinate system at the viewing end of the imagecapturing device;

FIG. 13 shows a schematic block diagram indicating steps followed by thecontrol system of the minimally invasive surgical apparatus indetermining pincher formation position and orientation relative to theeye Cartesian reference coordinate system;

FIG. 14 shows a block diagram representing control steps followed by thecontrol system of the minimally invasive surgical apparatus in effectingcontrol between pincher formation positional and orientational movementand end effector positional and orientational movement;

FIG. 15 shows further detail of a “simulated domain” of the controlsystem shown in FIG. 14;

FIG. 16 shows one embodiment of a simulation block shown in FIG. 15;

FIG. 17 shows a relationship between L and 1/L;

FIG. 18 shows another embodiment of the simulation block shown in FIG.15;

FIG. 19 shows a block diagram indicating the imposition of simulatedvelocity and position limits;

FIG. 20 shows a preferred embodiment of the simulation block shown inFIG. 15;

FIG. 21 shows a block diagram indicating the imposition of simulatedvelocity and position limits relating to orientational slave movement;

FIG. 22 shows a block diagram indicating control steps relating toaligning master position and orientation with that of the slave;

FIG. 23 shows a block diagram indicating further detail of a masteralignment step indicated in FIG. 22;

FIG. 24 shows a block diagram indicating further detail of a masterinverse kinematics step shown in FIG. 23;

FIG. 25 shows a block diagram indicating further detail of an absolutemaster wrist alignment step shown in FIG. 24;

FIG. 26 shows a schematic diagram illustrating further detail relatingto FIG. 25;

FIG. 27 shows a block diagram indicating further detail of anincremental master wrist alignment step shown in FIG. 24; and

FIGS. 28 to 30 show three-dimensional diagrams of different wrist gimbalconfigurations having the same pincher formation orientation.

DESCRIPTION OF THE SPECIFIC EMBODIMENTS

Referring to FIG. 1A of the drawings, an operator station or surgeon'sconsole of a minimally invasive telesurgical system is generallyindicated by reference numeral 200. The station 200 includes a viewer202 where an image of a surgical site is displayed in use. A support 204is provided on which an operator, typically a surgeon, can rest his orher forearms while gripping two master controls (not shown in FIG. 1A),one in each hand. The master controls are positioned in a space 206inwardly beyond the support 204. When using the control station 200, thesurgeon typically sits in a chair in front of the control station 200,positions his or her eyes in front of the viewer 202 and grips themaster controls one in each hand while resting his or her forearms onthe support 204.

In FIG. 1B of the drawings, a cart or surgical station of thetelesurgical system is generally indicated by reference numeral 300. Inuse, the cart 300 is positioned close to a patient requiring surgery andis then normally caused to remain stationary until a surgical procedureto be performed has been completed. The cart 300 typically has wheels orcastors to render it mobile. The station 200 is typically positionedremote from the cart 300 and can be separated from the cart 300 by agreat distance, even miles away, but will typically be used within anoperating room with the cart 300.

The cart 300 typically carries three robotic arm assemblies. One of therobotic arm assemblies, indicated by reference numeral 302, is arrangedto hold an image capturing device 304, e.g., an endoscope, or the like.Each of the two other arm assemblies 10, 10 respectively, includes asurgical instrument 14. The endoscope 304 has a viewing end 306 at aremote end of an elongate shaft thereof. It will be appreciated that theendoscope 304 has an elongate shaft to permit its viewing end 306 to beinserted through an entry port into an internal surgical site of apatient's body. The endoscope 304 is operatively connected to the viewer202 to display an image captured at its viewing end 306 on the viewer202. Each robotic arm assembly 10, 10 is normally operatively connectedto one of the master controls. Thus, the movement of the robotic armassemblies 10, 10 is controlled by manipulation of the master controls.The instruments 14 of the robotic arm assemblies 10, 10 have endeffectors which are mounted on wrist members which are pivotally mountedon distal ends of elongate shafts of the instruments 14, as is describedin greater detail hereinbelow. It will be appreciated that theinstruments 14 have elongate shafts to permit the end effectors to beinserted through entry ports into the internal surgical site of apatient's body. Movement of the end effectors relative to the ends ofthe shafts of the instruments 14 is also controlled by the mastercontrols.

The robotic arms 10, 10, 302 are mounted on a carriage 97 by means ofsetup joint arms 95. The carriage 97 can be adjusted selectively to varyits height relative to a base 99 of the cart 300, as indicated by arrowsK. The setup joint arms 95 are arranged to enable the lateral positionsand orientations of the arms 10, 10, 302 to be varied relative to avertically extending column 93 of the cart 300. Accordingly, thepositions, orientations and heights of the arms 10, 10, 302 can beadjusted to facilitate passing the elongate shafts of the instruments 14and the endoscope 304 through the entry ports to desired positionsrelative to the surgical site. When the surgical instruments 14 andendoscope 304 are so positioned, the setup joint arms 95 and carriage 97are typically locked in position.

In FIGS. 2A and 2B of the drawings, one of the robotic arm assemblies 10is shown in greater detail. Each assembly 10 includes an articulatedrobotic arm 12, and a surgical instrument, schematically and generallyindicated by reference numeral 14, mounted thereon. FIG. 3 indicates thegeneral appearance of the surgical instrument 14 in greater detail.

The surgical instrument 14 includes an elongate shaft 14.1. Thewrist-like mechanism, generally indicated by reference numeral 50, islocated at a working end of the shaft 14.1. A housing 53, arrangedreleasably to couple the instrument 14 to the robotic arm 12, is locatedat an opposed end of the shaft 14.1. In FIG. 2A, and when the instrument14 is coupled or mounted on the robotic arm 12, the shaft 14.1 extendsalong an axis indicated at 14.2. The instrument 14 is typicallyreleasably mounted on a carriage 11, which can be driven to translatealong a linear guide formation 24 of the arm 12 in the direction ofarrows P.

The robotic arm 12 is typically mounted on a base or platform at an endof its associated setup joint arm 95 by means of a bracket or mountingplate 16.

The robotic arm 12 includes a cradle, generally indicated at 18, anupper arm portion 20, a forearm portion 22 and the guide formation 24.The cradle 18 is pivotally mounted on the plate 16 in a gimbaled fashionto permit rocking movement of the cradle 18 in the direction of arrows26 as shown in FIG. 2B, about a pivot axis 28. The upper arm portion 20includes link members 30, 32 and the forearm portion 22 includes linkmembers 34, 36. The link members 30, 32 are pivotally mounted on thecradle 18 and are pivotally connected to the link members 34, 36. Thelink members 34, 36 are pivotally connected to the guide formation 24.The pivotal connections between the link members 30, 32, 34, 36, thecradle 18, and the guide formation 24 are arranged to constrain therobotic arm 12 to move in a specific manner. The movement of the roboticarm 12 is illustrated schematically in FIG. 4.

With reference to FIG. 4, the solid lines schematically indicate oneposition of the robotic arm and the dashed lines indicate anotherpossible position into which the arm can be displaced from the positionindicated in solid lines.

It will be understood that the axis 14.2 along which the shaft 14.1 ofthe instrument 14 extends when mounted on the robotic arm 12 pivotsabout a pivot center or fulcrum 49. Thus, irrespective of the movementof the robotic arm 12, the pivot center 49 normally remains in the sameposition relative to the stationary cart 300 on which the arm 12 ismounted. In use, the pivot center 49 is positioned at a port of entryinto a patient's body when an internal surgical procedure is to beperformed. It will be appreciated that the shaft 14.1 extends throughsuch a port of entry, the wrist-like mechanism 50 then being positionedinside the patient's body. Thus, the general position of the mechanism50 relative to the surgical site in a patient's body can be changed bymovement of the arm 12. Since the pivot center 49 is coincident with theport of entry, such movement of the arm does not excessively effect thesurrounding tissue at the port of entry.

As can best be seen with reference to FIG. 4, the robotic arm 12provides three degrees of freedom of movement to the surgical instrument14 when mounted thereon. These degrees of freedom of movement arefirstly the gimbaled motion indicated by arrows 26, pivoting or pitchingmovement as indicated by arrows 27 and the linear displacement in thedirection of arrows P. Movement of the arm as indicated by arrows 26, 27and P is controlled by appropriately positioned actuators, e.g.,electrical motors, or the like, which respond to inputs from itsassociated master control to drive the arm 12 to a required position asdictated by movement of the master control. Appropriately positionedsensors, e.g., potentiometers, encoders, or the like, are provided onthe arm and its associated setup joint arm 95 to enable a control systemof the minimally invasive telesurgical system to determine jointpositions, as described in greater detail hereinbelow. It will beappreciated that whenever “sensors” are referred to in thisspecification, the term is to be interpreted widely to include anyappropriate sensors such as positional sensors, velocity sensors, or thelike. It will be appreciated that by causing the robotic arm 12selectively to displace from one position to another, the generalposition of the wrist-like mechanism 50 at the surgical site can bevaried during the performance of a surgical procedure.

Referring now to FIG. 5 of the drawings, the wrist-like mechanism 50will now be described in greater detail. In FIG. 5, the working end ofthe shaft 14.1 is indicated at 14.3. The wrist-like mechanism 50includes a wrist member 52. One end portion of the wrist member 52 ispivotally mounted in a clevis, generally indicated at 17, on the end14.3 of the shaft 14.1 by means of a pivotal connection 54. The wristmember 52 can pivot in the direction of arrows 56 about the pivotalconnection 54. An end effector, generally indicated by reference numeral58, is pivotally mounted on an opposed end of the wrist member 52. Theend effector 58 is in the form of, e.g., a clip applier for anchoringclips during a surgical procedure. Accordingly, the end effector 58 hastwo parts 58.1, 58.2 together defining a jaw-like arrangement.

It will be appreciated that the end effector can be in the form of anyrequired surgical tool, e.g., having two members or fingers which pivotrelative to each other, such as scissors, pliers for use as needledrivers, or the like. Instead, it can include a single working member,e.g., a scalpel, cautery electrode, or the like. When a tool other thana clip applier is required during the surgical procedure, the tool 14 issimply removed from its associated arm and replaced with an instrumentbearing the required end effector, e.g., a scissors, or pliers, or thelike.

The end effector 58 is pivotally mounted in a clevis, generallyindicated by reference numeral 19, on an opposed end of the wrist member52, by means of a pivotal connection 60. It will be appreciated thatfree ends 11, 13 of the parts 58.1, 58.2 are angularly displaceableabout the pivotal connection 60 toward and away from each other asindicated by arrows 62, 63. It will further be appreciated that themembers 58.1, 58.2 can be displaced angularly about the pivotalconnection 60 to change the orientation of the end effector 58 as awhole, relative to the wrist member 52. Thus, each part 58.1, 58.2 isangularly displaceable about the pivotal connection 60 independently ofthe other, so that the end effector 58, as a whole, is angularlydisplaceable about the pivotal connection 60 as indicated in dashedlines in FIG. 5. Furthermore, the shaft 14.1 is rotatably mounted on thehousing 53 for rotation as indicated by the arrows 59. Thus, the endeffector 58 has three degrees of freedom of movement relative to the arm12, namely, rotation about the axis 14.2 as indicated by arrows 59,angular displacement as a whole about the pivot 60 and angulardisplacement about the pivot 54 as indicated by arrows 56. By moving theend effector within its three degrees of freedom of movement, itsorientation relative to the end 14.3 of the shaft 14.1 can selectivelybe varied. It will be appreciated that movement of the end effectorrelative to the end 14.3 of the shaft 14.1 is controlled byappropriately positioned actuators, e.g., electrical motors, or thelike, which respond to inputs from the associated master control todrive the end effector 58 to a required orientation as dictated bymovement of the master control. Furthermore, appropriately positionedsensors, e.g., encoders, or potentiometers, or the like, are provided topermit the control system of the minimally invasive telesurgical systemto determine joint positions as described in greater detail hereinbelow.

One of the master controls 700, 700 is indicated in FIG. 6C of thedrawings. A hand held part or wrist gimbal of the master control device700 is indicated in FIG. 6A and is generally indicated by referencenumeral 699. Part 699 has an articulated arm portion including aplurality of members or links 702 connected together by pivotalconnections or joints 704. The surgeon grips the part 699 by positioninghis or her thumb and index finger over a pincher formation 706. Thesurgeon's thumb and index finger are typically held on the pincherformation 706 by straps (not shown) threaded through slots 710. When thepincher formation 706 is squeezed between the thumb and index finger,the fingers or end effector elements of the end effector 58 close. Whenthe thumb and index finger are moved apart the fingers of the endeffector 58 move apart in sympathy with the moving apart of the pincherformation 706. The joints of the part 699 are operatively connected toactuators, e.g., electric motors, or the like, to provide for, e.g.,force feedback, gravity compensation, and/or the like, as described ingreater detail hereinbelow. Furthermore, appropriately positionedsensors, e.g., encoders, or potentiometers, or the like, are positionedon each joint 704 of the part 699, so as to enable joint positions ofthe part 699 to be determined by the control system.

The part 699 is typically mounted on an articulated arm 712 as indicatedin FIG. 6B. Reference numeral 4 in FIGS. 6A and 6B indicates thepositions at which the part 699 and the articulated arm 712 areconnected together. When connected together, the part 699 can displaceangularly about an axis at 4.

The articulated arm 712 includes a plurality of links 714 connectedtogether at pivotal connections or joints 716. It will be appreciatedthat also the articulated arm 712 has appropriately positionedactuators, e.g., electric motors, or the like, to provide for, e.g.,force feedback, gravity compensation, and/or the like. Furthermore,appropriately positioned sensors, e.g., encoders, or potentiometers, orthe like, are positioned on the joints 716 so as to enable jointpositions of the articulated arm 712 to be determined by the controlsystem as described in greater detail hereinbelow.

To move the orientation of the end effector 58 and/or its position alonga translational path, the surgeon simply moves the pincher formation 706to cause the end effector 58 to move to where he wants the end effector58 to be in the image viewed in the viewer 202. Thus, the end effectorposition and/or orientation is caused to follow that of the pincherformation 706.

The master control devices 700, 700 are typically mounted on the station200 through pivotal connections at 717 as indicated in FIG. 6B. Asmentioned hereinbefore, to manipulate each master control device 700,the surgeon positions his or her thumb and index finger over the pincherformation 706. The pincher formation 706 is positioned at a free end ofthe part 699 which in turn is mounted on a free end of the articulatedarm portion 712.

The electric motors and sensors associated with the robotic arms 12 andthe surgical instruments 14 mounted thereon, and the electric motors andsensors associated with the master control devices 700 are operativelylinked in the control system. The control system typically includes atleast one processor, typically a plurality of processors, for effectingcontrol between master control device input and responsive robotic armand surgical instrument output and for effecting control between roboticarm and surgical instrument input and responsive master control outputin the case of, e.g., force feedback.

In use, and as schematically indicated in FIG. 7 of the drawings, thesurgeon views the surgical site through the viewer 202. The end effector58 carried on each arm 12 is caused to perform positional andorientational movements in response to movement and action inputs on itsassociated master controls. The master controls are indicatedschematically at 700, 700. It will be appreciated that during a surgicalprocedure images of the end effectors 58 are captured by the endoscope304 together with the surgical site and are displayed on the viewer 202so that the surgeon sees the responsive movements and actions of the endeffectors 58 as he or she controls such movements and actions by meansof the master control devices 700, 700. The control system is arrangedto cause end effector orientational and positional movement as viewed inthe image at the viewer 202 to be mapped onto orientational andpositional movement of a pincher formation of the master control as willbe described in greater detail hereinbelow.

The operation of the control system of the minimally invasive surgicalapparatus will now be described in greater detail. In the descriptionwhich follows, the control system will be described with reference to asingle master control 700 and its associated robotic arm 12 and surgicalinstrument 14. The master control 700 will be referred to simply as“master” and its associated robotic arm 12 and surgical instrument 14will be referred to simply as “slave.”

The method whereby control between master movement and correspondingslave movement is achieved by the control system of the minimallyinvasive surgical apparatus will now be described with reference toFIGS. 7 to 9 of the drawings in overview fashion. The method will thenbe described in greater detail with reference to FIGS. 10 to 21 of thedrawings.

In essence, control between master and slave movement is achieved bycomparing master position and orientation in an eye Cartesian coordinatereference system with slave position and orientation in a cameraCartesian coordinate reference system. For ease of understanding andeconomy of words, the term “Cartesian coordinate reference system” willsimply be referred to as “frame” in the rest of this specification.Accordingly, when the master is stationary, the slave position andorientation within the camera frame is compared with the master positionand orientation in the eye frame, and should the position and/ororientation of the slave in the camera frame not correspond with theposition and/or orientation of the master in the eye frame, the slave iscaused to move to a position and/or orientation in the camera frame atwhich its position and/or orientation in the camera frame doescorrespond with the position and/or orientation of the master in the eyeframe. In FIG. 8, the camera frame is generally indicated by referencenumeral 610 and the eye frame is generally indicated by referencenumeral 612 in FIG. 9.

When the master is moved into a new position and/or orientation in theeye frame 612, the new master position and/or orientation does notcorrespond with the previously corresponding slave position and/ororientation in the camera frame 610. The control system then causes theslave to move into a new position and/or orientation in the camera frame610 at which new position and/or orientation, its position andorientation in the camera frame 610 does correspond with the newposition and/or orientation of the master in the eye frame 612.

It will be appreciated that the control system includes at least one,and typically a plurality, of processors which compute new correspondingpositions and orientations of the slave in response to master movementinput commands on a continual basis determined by the processing cyclerate of the control system. A typical processing cycle rate of thecontrol system under discussion is about 1300 Hz. Thus, when the masteris moved from one position to a next position, the correspondingmovement required by the slave to respond is computed at about 1300 Hz.Naturally, the control system can have any appropriate processing cyclerate depending on the processor or processors used in the controlsystem.

The camera frame 610 is positioned such that its origin 614 ispositioned at the viewing end 306 of the endoscope 304. Conveniently,the z axis of the camera frame 610 extends axially along a viewing axis616 of the endoscope 304. Although in FIG. 8, the viewing axis 616 isshown in coaxial alignment with a shaft axis of the endoscope 304, it isto be appreciated that the viewing axis 616 can be angled relativethereto. Thus, the endoscope can be in the form of an angled scope.Naturally, the x and y axes are positioned in a plane perpendicular tothe z axis. The endoscope is typically angularly displaceable about itsshaft axis. The x, y and z axes are fixed relative to the viewing axisof the endoscope 304 so as to displace angularly about the shaft axis insympathy with angular displacement of the endoscope 304 about its shaftaxis.

To enable the control system to determine slave position andorientation, a frame is defined on or attached to the end effector 58.This frame is referred to as an end effector frame or slave tip frame,in the rest of this specification, and is generally indicated byreference numeral 618. The end effector frame 618 has its origin at thepivotal connection 60. Conveniently, one of the axes e.g. the z axis, ofthe frame 618 is defined to extend along an axis of symmetry, or thelike, of the end effector 58. Naturally, the x and y axes then extendperpendicularly to the z axis. It will appreciated that the orientationof the slave is then defined by the orientation of the frame 618 havingits origin at the pivotal connection 60, relative to the camera frame610. Similarly, the position of the slave is then defined by theposition of the origin of the frame at 60 relative to the camera frame610.

Referring now to FIG. 9 of the drawings, the eye frame 612 is chosensuch that its origin corresponds with a position 201 where the surgeon'seyes are normally located when he or she is viewing the surgical site atthe viewer 202. The z axis extends along a line of sight of the surgeon,indicated by axis 620, when viewing the surgical site through the viewer202. Naturally, the x and y axes extend perpendicularly from the z axisat the origin 201. Conveniently, the y axis is chosen to extendgenerally vertically relative to the viewer 202 and the x axis is chosento extend generally horizontally relative to the viewer 202.

To enable the control system to determine master position andorientation within the viewer frame 612, a point on the master is chosenwhich defines an origin of a master or master tip frame, indicated byreference numeral 622. This point is chosen at a point of intersectionindicated by reference numeral 3A between axes of rotation 1 and 3 ofthe master, as can best be seen in FIG. 6A of the drawings.Conveniently, the z axis of the master frame 622 on the master extendsalong an axis of symmetry of the pincher formation 706 which extendscoaxially along the rotational axis 1. The x and y axes then extendperpendicularly from the axis of symmetry 1 at the origin 3A.Accordingly, orientation of the master within the eye frame 612 isdefined by the orientation of the master frame 622 relative to the eyeframe 612. The position of the master in the eye frame 612 is defined bythe position of the origin 3A relative to the eye frame 612.

How the position and orientation of the slave within the camera frame610 is determined by the control system will now be described withreference to FIG. 10 of the drawings. FIG. 10 shows a schematic diagramof one of the robotic arm 12 and surgical instrument 14 assembliesmounted on the cart 300. However, before commencing with a descriptionof FIG. 10, it is appropriate to describe certain previously mentionedaspects of the surgical station 300 which impact on the determination ofthe orientation and position of the slave relative to the camera frame610.

In use, when it is desired to perform a surgical procedure by means ofthe minimally invasive surgical apparatus, the surgical station 300 ismoved into close proximity to a patient requiring the surgicalprocedure. The patient is normally supported on a surface such as anoperating table, or the like. To make allowance for support surfaces ofvarying height, and to make allowance for different positions of thesurgical station 300 relative to the surgical site at which the surgicalprocedure is to be performed, the surgical station 300 is provided withthe ability to have varying initial setup configurations. Accordingly,the robotic arms 12, 12, and the endoscope arm 302 are mounted on thecarriage 97 which is heightwise adjustable, as indicated by arrows K,relative to the base 99 of the cart 300, as can best be seen in FIGS. 1Band 10 of the drawings. Furthermore, the robotic arms 12, 12 and theendoscope arm 302 are mounted on the carriage 97 by means of the setupjoint arms 95. Thus, the lateral position and orientation of the arms12, 12, 302 can be selected by moving the setup joint arms 95. Thus, atthe commencement of the surgical procedure, the cart 300 is moved intothe position in close proximity to the patient, an appropriate height ofthe carriage 97 is selected by moving it to an appropriate heightrelative to the base 99 and the surgical instruments 14 are movedrelative to the carriage 97 so as to introduce the shafts of theinstruments 14 and the endoscope 304 through the ports of entry and intopositions in which the end effectors 58 and the viewing end 306 of theendoscope 304 are appropriately positioned at the surgical site and thefulcrums are coincident with the ports of entry. Once the height andpositions are selected, the carriage 97 is locked at its appropriateheight and the setup joint arms 95 are locked in their positions andorientations. Normally, throughout the surgical procedure, the carriage97 is maintained at the selected height and similarly the setup jointarms 95 are maintained in their selected positions. However, ifrequired, either the endoscope or one or both of the instruments can beintroduced through other ports of entry during the surgical procedure.

Returning now to FIG. 10, the determination by the control system of theposition and orientation of the slave within the camera frame 610 willnow be described. It will be appreciated that this is achieved by meansof one or more processors having a specific processing cycle rate. Thus,where appropriate, whenever position and orientation are referred to inthis specification, it should be borne in mind that a correspondingvelocity is also readily determined. The control system determines theposition and orientation of the slave within the camera frame 610 bydetermining the position and orientation of the slave relative to a cartframe 624 and by determining the orientation and position of theendoscope 304 with reference to the same cart frame 624. The cart frame624 has an origin indicated by reference numeral 626 in FIG. 10.

To determine the position and orientation of the slave relative to thecart frame 624, the position of a fulcrum frame 630 having its origin atthe fulcrum 49 is determined within the cart frame 624 as indicated bythe arrow 628 in dashed lines. It will be appreciated that the positionof the fulcrum 49 normally remains at the same location, coincident witha port of entry into the surgical site, throughout the surgicalprocedure. The position of the end effector frame 618 on the slave,having its origin at the pivotal connection 60, is then determinedrelative to the fulcrum frame 630 and the orientation of the endeffector frame 618 on the slave is also determined relative to thefulcrum frame 630. The position and orientation of the end effectorframe 618 relative to the cart frame is then determined by means ofroutine calculation using trigonometric relationships.

It will be appreciated that the robotic arm 302 of the endoscope 304 isconstrained to move in similar fashion to the robotic arm 10, asindicated schematically in FIG. 4 of the drawings. Thus, the endoscope304 when positioned with its viewing end 306 directed at the surgicalsite, also defines a fulcrum coincident with its associated port ofentry into the surgical site. The endoscope arm 302 can be driven tocause the endoscope 304 to move into a different position during asurgical procedure, to enable the surgeon to view the surgical site froma different position in the course of performing the surgical procedure.It will be appreciated that movement of the viewing end 306 of theendoscope 304 is performed by varying the orientation of the endoscope304 relative to its pivot center or fulcrum. The position andorientation of the camera frame 610 within the cart frame 624 isdetermined in similar fashion to the position and orientation of theslave within the cart frame 624. When the position and orientation ofthe camera frame 610 relative to the cart frame 624, and the positionand orientation of the slave relative to the cart frame 624 have beendetermined in this manner, the position and the orientation of the slaverelative to the camera frame 610 is readily determinable through routinecalculation using trigonometric relationships.

How the position and orientation of the master within the viewer frame612 is determined by the control system will now be described withreference to FIG. 11 of the drawings. FIG. 11 shows a schematic diagramof one of the master controls 700 at the operator station 200.

The operator station 200 optionally also includes setup joint arms, asindicated at 632, to enable the general location of the masters 700, 700to be varied to suit the surgeon. Thus, the general position of themasters 700, 700 can be selectively varied to bring the masters 700, 700into a general position at which they are comfortably positioned for thesurgeon. When the masters 700, 700 are thus comfortably positioned, thesetup joint arms 632 are locked in position and are normally maintainedin that position throughout the surgical procedure.

To determine the position and orientation of the master 700, asindicated in FIG. 11, within the eye frame 612, the position andorientation of the eye frame 612 relative to a surgeon's station frame634, and the position and orientation of the master 700 relative to thesurgeon's frame 634 is determined. The surgeon's station frame 634 hasits origin at a location which is normally stationary during thesurgical procedure, and is indicated at 636.

To determine the position and orientation of the master 700 relative tothe station frame 634, a position of a master setup frame 640 at an endof the setup joint arms 632 on which the master 700 is mounted, relativeto the station frame 636, is determined, as indicated by the arrow 638in dashed lines. The position and orientation of the master frame 622 onthe master 700 having its origin at 3A is then determined relative tothe master setup frame 640. In this manner, the position and orientationof the master frame 622 relative to the frame 634 can be determined bymeans of routine calculation using trigonometric relationships. Theposition and orientation of the eye frame 612 relative to the stationframe 634 is determined in similar fashion. It will be appreciated thatthe position of the viewer 202 relative to the rest of the surgeon'sconsole 200 can selectively be varied to suit the surgeon. The positionand orientation of the master frame 622 relative to the eye frame 612can then be determined from the position and orientation of the masterframe 622 and the eye frame 612 relative to the surgeon station frame634 by means of routine calculation using trigonometric relationships.

In the manner described above, the control system of the minimallyinvasive surgical apparatus determines the position and orientation ofthe end effector 58 by means of the end effector frame 618 in the cameraframe 610, and, likewise, determines the position and orientation of themaster by means of the master frame 622 relative to the eye frame 612.

As mentioned, the surgeon grips the master by locating his or her thumband index finger over the pincher formation 706. When the surgeon'sthumb and index finger are located on the pincher formation, the pointof intersection 3A is positioned inwardly of the thumb and index fingertips. The master frame having its origin at 3A is effectively mappedonto the end effector frame 618, having its origin at the pivotalconnection 60 of the end effector 58 as viewed by the surgeon in theviewer 202. Thus, when performing the surgical procedure, and thesurgeon manipulates the position and orientation of the pincherformation 706 to cause the position and orientation of the end effector58 to follow, it appears to the surgeon that his or her thumb and indexfinger are mapped onto the fingers of the end effector 58 and that thepivotal connection 60 of the end effector 58 corresponds with a virtualpivot point of the surgeon's thumb and index finger inwardly from thetips of the thumb and index finger. It will be appreciated thatdepending upon the actual configuration of the pincher formation, inparticular the point of intersection of the axes 1 and 3 relative to theposition of the pincher formation 706, the frame 622 on the master 700can be offset from the intersection 3A so as to approach a pointrelative to the surgeon's hand at which point the pivotal connection 60approximately corresponds.

Accordingly, as the surgical procedure is being performed the positionand orientation of the fingers of the end effector tracks orientationand position changes of the surgeon's thumb and index finger in anatural intuitive or superimposed fashion. Furthermore, actuation of theend effector 58, namely causing the end effector fingers selectively toopen and close, corresponds intuitively to the opening and closing ofthe surgeon's thumb and index finger. Thus, actuation of the endeffector 58 as viewed in the viewer 302 is performed by the surgeon in anatural intuitive manner, since the pivot point 60 of the end effector58 is appropriately mapped onto a virtual pivot point between thesurgeon's thumb and index finger.

It will be appreciated that the end effector frame 618 can, whereappropriate, be offset relative to the pivotal connection 60. Thus, forexample, should the end effector have fingers of a relatively longlength, the origin of the end effector frame can be offset in adirection toward the end effector finger tips. It will also beappreciated that using positional and/or orientational offsets betweenthe master frame 622 and the intersection 3A, as well as between the endeffector frame 618 and the pivotal connection 60, the mapping of thepincher formation 706 onto the end effector 58 may be shifted, forexample to map the tips of the pincher formation onto the tips of theend effector.

It will be appreciated that the cart frame 624 can be chosen at anyconvenient location in which its origin corresponds with a location onthe cart 300 which does not vary relative to its base 99. The surgeon'sstation frame 634 can likewise be chosen at any convenient location suchthat its origin is located at a position which does not vary relative toa base 642 thereof. Furthermore, to determine the position andorientation of the camera frame 610 relative to the cart frame 624, usecan be made of a plurality of different intermediate frame paths. Todetermine the position and orientation of the end effector frame 618relative to the cart frame 624 use can also be made of a plurality ofdifferent intermediate frame paths.

However, it has been found that should the intermediate frame paths beappropriately selected, the control system is then arranged to bereadily adaptable to accommodate modular replacement of modular partshaving different characteristics than the modular parts being replaced.It will be appreciated that selecting intermediate frames also eases thecomputational process involved in determining master and slave positionand orientation.

Referring again to FIG. 10 of the drawings, the cart frame is chosen at624, as already mentioned. It will be appreciated that determining theposition of the fulcrum frame 630 relative to the cart frame 624 isachieved through appropriately positioned sensors, such aspotentiometers, encoders, or the like. Conveniently, the fulcrum frameposition 630 relative to the cart frame 624 is determined through twointermediate frames. One of the frames is a carriage guide frame 644which has its origin at a convenient location on a guide along which thecarriage 97 is guided. The other frame, an arm platform frame indicatedat 646 is positioned at an end of the setup joint arm 95 on which therobotic arm 12 is mounted. Thus, when slave position and orientation isdetermined relative to the cart frame 624, the carriage guide frame 644position relative to the cart frame 624 is determined, then the platformframe 646 position relative to the carriage guide frame 644, then thefulcrum frame 630 relative to the platform frame 646, and then the slaveorientation and position relative to the fulcrum frame 630, thereby todetermine the slave position and orientation relative to the cart frame624. It will be appreciated that the slave position and orientationrelative to the cart frame 624 is determined in this manner for each arm10 and in similar fashion for the camera frame 610, through its arm 302,relative to the cart frame 624.

Referring to FIG. 11, the position and orientation of the master controlis determined by determining the position of a base frame 648 relativeto the surgeon's station frame 634, then determining the position of theplatform frame 640 relative to the base frame 648, and then determiningmaster position and orientation relative to the platform frame 640. Theposition and orientation of the master frame 622 relative to thesurgeon's station frame 634 is then readily determined through routinecalculation using trigonometric relationships. It will be appreciatedthat the position and orientation of the other master frame relative tothe surgeon console frame 634 is determined in a similar fashion.

Referring to FIG. 10, by choosing the frames as described, the setupjoint 95 can be replaced with another setup joint while the same roboticarm is used. The control system can then be programmed with information,e.g., arm lengths and/or the like, relating to the new setup joint only.Similarly, the robotic arm 10 can be replaced with another arm, thecontrol system then requiring programming with information, e.g.,fulcrum position and/or the like, relating to the new robotic arm only.It will be appreciated that in this way the endoscope arm 302 and itsassociated setup joint can also be independently replaced, the controlsystem then requiring programming of information relating only to thepart being replaced. Furthermore, referring to FIG. 11, the setup jointand master control can also independently be replaced, the controlsystem requiring programming of information relating to thecharacteristics of the new part only.

Further aspects of the control system of the minimally invasive surgicalapparatus will now be described with reference to FIG. 12.

FIG. 12 indicates the control steps whereby the control system of theminimally invasive surgical apparatus determines slave position andorientation, namely the position and orientation of the end effectorframe 618 in the camera frame 610.

The position or offsets of the carriage guide frame 644 relative to thecart frame 624 is indicated at 621. The offsets at 621 are fed through aforward kinematics block (FKIN) at 623 to yield corresponding Cartesiancoordinates of the frame 644 relative to the cart frame 624.

Sensors 625 operatively associated with the setup joint arm 95 andsensors determining the height of the carriage 97, are read by aprocessor 627 to determine translational and joint positions. Thetranslational and joint positions are then input to an FKIN block 629 todetermine corresponding Cartesian coordinates. At 631, the Cartesiancoordinates of the carriage guide frame 644 relative to the cart frame624 and the Cartesian coordinates of the platform frame 646 relative tothe carriage frame 644 are used to determine the Cartesian coordinatesof the platform frame 646 relative to the cart frame 624.

Since the position of the fulcrum 49 relative to the platform frame 646does not change, an offset relative to the platform frame 646, indicatedat 633, is input to an FKIN controller at 635 to yield Cartesiancoordinates of the fulcrum frame 630 relative to the platform frame 646.It will be appreciated that, where appropriate, the term FKIN controlleris to be interpreted to include an appropriate conversion matrix andkinematic relationships. At 637, the Cartesian coordinates of thefulcrum frame 630 relative to the cart frame 624 are determined by meansof the values determined at 631 and 635 respectively.

It will be appreciated that, in similar fashion, the Cartesiancoordinates of the fulcrum of the endoscope is determined relative tothe cart frame 624. This is indicated at 639.

As mentioned, the position and orientation of the endoscope 304 can bevaried. The position and orientation of the endoscope 304 can be variedduring set up of the cart 300 before the surgical procedure commences orduring the performance of a surgical procedure should the surgeon wishto view the surgical site from a different location.

To enable the control system to determine endoscope position andorientation relative to the cart frame 624, sensors are provided on itsassociated arm 302. These sensors, indicated at 641, are read by aprocessor at 643 to determine joint positions. The joint positions thusdetermined are fed to an FKIN controller at 645, together with theCartesian coordinates determined at 639 to determine endoscopeorientation and position relative to the cart frame 624. These valuesare then input to 647 together with the values determined at 637, so asto enable the fulcrum frame 630 of the slave to be determined relativeto the camera frame 610.

During the course of the surgical procedure, the slave orientation andposition is normally constantly changing. Varying joint positions andvelocities are fed into an FKIN controller at 653, together with theCartesian coordinate values of the slave position relative to the cameraframe determined at 647 to yield Cartesian position and velocity of theslave, namely the end effector frame 618, relative to the camera frame610, as indicated by arrows 655, 657 respectively. For economy of words,Cartesian position is to be interpreted to include Cartesian orientationin the rest of this specification where appropriate. The varying jointpositions and velocities are fed into the FKIN block 653 from asimulation domain as described in greater detail hereinbelow.

Referring now to FIG. 13, master position and orientation relative tothe viewer frame 612 will now be described.

The base frame 648 normally does not change relative to the surgeonstation frame 634. Similarly, the frame at 640 normally does not changerelative to the base frame 648. As mentioned, setup joints canoptionally be provided at 632 if required. For the sake of thedescription which follows, the position of the frame at 640 relative tothe base frame 648 is assumed to be unchangeable. Naturally, if setupjoint arms are provided at 632, appropriate sensors would then beprovided to enable the position of the frame at 640 to be determinedrelative to the frame at 648.

Referring now to FIG. 13, offsets determining the frame 648 positionrelative to the surgeon station frame 634, as indicated at 659, are fedthrough an FKIN controller 665 to yield Cartesian coordinates of thebase frame 648 relative to the surgeon station frame 634. Similarly,offsets relating to frame 640 position relative to base frame 648position, as indicated at 661, are fed through an FKIN controller at 663to yield Cartesian coordinates of the frame 640 relative to the baseframe 648. From the values derived at 665, 663, the Cartesiancoordinates of the frame 640 relative to the surgeon station frame 634are determined at 667.

Offsets at 697 relating to a viewer base frame, not indicated in FIG.11, are fed through an FKIN controller at 669 to yield correspondingCartesian coordinates of the base frame relative to the frame 634. Theviewer 202 can be positionally adjustable relative to the rest of theoperator station 200. To enable a viewer position relative to the viewerbase frame to be determined, appropriately positioned sensors 671 areprovided. Sensor readings from these sensors at 671 are processed at 673to determine joint or translational positions which are then fed throughan FKIN controller at 675 to yield Cartesian coordinates of the viewerframe relative to the viewer base frame. At 677, the viewer frameposition in Cartesian coordinates relative to the surgeon station frame634 are determined from the values derived at 669 and 675 respectively.

Offsets corresponding to the position of the surgeon's eyes relative tothe viewer frame at 679 are fed through an FKIN controller at 681 toyield Cartesian coordinates of the position of the surgeon's eyesrelative to the viewer frame. At 683, the values from 677 and 681 areused to determine the surgeon's eye frame 612 relative to the surgeonstation frame 634.

At 685, the values from 667 and 683 are used to determine the positionof the frame 640 relative to the eye frame 612.

Naturally, master position and orientation relative to the eye frame 612is continually changing during the course of a surgical procedure. Thesensors on the master 700, indicated at 687, are read by a processor at689 to determine master joint position and velocity. These jointposition and velocity values are then fed through an FKIN controller at691, together with the value derived at 685 to yield master Cartesianposition and velocity values 693, 695 relating to Cartesian position andvelocity of master frame 622, relative to the eye frame 612.

At the commencement of a surgical procedure, an initial position of themaster 700 is set to correspond with an initial position of the slave.Thereafter, as the master 700 is moved, the control system monitors suchmovement and commands the slave to track the master movement. Thus, atthe commencement of a surgical procedure, the frame 618 on the slave atthe pivotal connection 60, relative to its reference frame 610 at theviewing end 306 of the endoscope 304, at the initial position, is mappedonto the master frame 622 relative to its reference eye frame 612 at itsinitial position. Similarly, the system maps an initial orientation ofthe pincher formation frame 622 with an initial orientation of the endeffector frame 618. Thus, the orientation of the end effector frame 618is also caused to track the orientation of the master frame 622. Theposition and orientation of the slave in the camera frame 610 need notcorrespond identically with the position and orientation of the masterin the eye frame 612. Accordingly, offsets can be introduced relating tothe orientation and the position of the end effector frame 618 relativeto the camera frame 610 to define an arbitrary end effector frameposition and orientation which corresponds to a master frame 622position and orientation in the eye frame 612. It will be appreciatedthat the control system can readily determine the orientation and theposition of the end effector frame 618 relative to the camera frame 610at which it is to correspond with that of the master frame relative tothe eye frame by means of the frames and offsets discussed above. Thus,even during the course of a surgical procedure, if the control betweenmaster and slave is interrupted and the endoscope is moved, or one orboth of the surgical instruments are repositioned through differentports of entry, or the master positions are changed at the surgeon'sconsole, or the like, re-mapping of slave relative to master in theirrespective camera and eye frames can readily be achieved by the controlsystem.

The control system, generally indicated by reference numeral 810, willnow be described in greater detail with reference to FIG. 14 of thedrawings, in which like reference numerals are used to designate similarparts or aspects, unless otherwise stated.

As mentioned earlier, the master control 700 has sensors, e.g.,encoders, or potentiometers, or the like, associated therewith to enablethe control system 810 to determine the position of the master control700 in joint space as it is moved from one position to a next positionon a continual basis during the course of performing a surgicalprocedure. In FIG. 14, signals from these positional sensors areindicated by arrow 814. Positional readings measured by the sensors at687 are read by the processor indicated at 689 (refer to FIG. 13). Itwill be appreciated that since the master control 700 includes aplurality of joints connecting one arm member thereof to the next,sufficient positional sensors are provided on the master 700 to enablethe angular position of each arm member relative to the arm member towhich it is joined to be determined thereby to enable the position andorientation of the master frame 622 on the master to be determined. Asthe angular positions of one arm member relative to the arm member towhich it is joined is read cyclically by the processor 689 in responseto movements induced on the master control 700 by the surgeon, theangular positions are continuously changing. The processor at 689 readsthese angular positions and computes the rate at which these angularpositions are changing. Thus, the processor 689 reads angular positionsand computes the rate of angular change, or joint velocity, on acontinual basis corresponding to the system processing cycle time, i.e.,1300 Hz. Joint position and joint velocity commands thus computed at 689are then input to the Forward Kinematics (FKIN) controller at 691, asalready described hereinabove.

At the FKIN controller 691, the positions and velocities in joint spaceare transformed into corresponding positions and velocities in Cartesianspace, relative to the eye frame 612 (refer to FIGS. 11 and 13). TheFKIN controller 691 is a processor which typically employs a Jacobian(J) matrix to accomplish this. It will be appreciated that the Jacobianmatrix transforms angular positions and velocities into correspondingpositions and velocities in Cartesian space by means of conventionaltrigonometric relationships. Thus, corresponding positions andvelocities in Cartesian space, or Cartesian velocity and positioncommands, are computed by the FKIN controller 691 which correspond toCartesian position and velocity changes of the master frame 622 in theeye frame 612.

The velocity and the position in Cartesian space is input into aCartesian controller, indicated at 820, and into a scale and offsetconverter, indicated at 822.

The minimally invasive surgical apparatus provides for a scale changebetween master control input movement and responsive slave outputmovement. Thus, a scale can be selected where, for example, a 1-inchmovement of the master control 700 is transformed into a correspondingresponsive ⅕-inch movement on the slave. At the scale and offset step822, the Cartesian position and velocity values are scaled in accordancewith the scale selected to perform the surgical procedure. Naturally, ifa scale of 1:1 has been selected, no change in scale is effected at 822.Similarly, offsets are taken into account which determine thecorresponding position and/or orientation of the end effector frame 618in the camera frame 610 relative to the position and orientation of themaster frame 622 in the eye frame 612.

After a scale and offset step is performed at 822, a resultant desiredslave position and desired slave velocity in Cartesian space is input toa simulated or virtual domain at 812, as indicated by arrows 811. Itwill be appreciated that the labeling of the block 812 as a simulated orvirtual domain is for identification only. Accordingly, the simulatedcontrol described hereinbelow is performed by elements outside the block812 also.

The simulated domain 812 will be described in greater detailhereinbelow. However, the steps imposed on the desired slave velocityand position in the virtual domain 812 will now be described broadly forease of understanding of the description which follows. In essence, acurrent slave position and velocity is continually monitored in thevirtual or simulated domain 812. The desired slave position and velocityis compared with the current slave position and velocity. Should thedesired slave position and/or velocity as input from 822 not causetransgression of limitations, e.g., velocity and/or position and/orsingularity, and/or the like, as set in the virtual domain 812, asimilar Cartesian slave velocity and position is output from the virtualdomain 812 and input into an inverse scale and offset converter asindicated at 826. The similar velocity and position output in Cartesianspace from the virtual domain 812 is indicated by arrows 813 andcorresponds with actual commands in joint space output from the virtualdomain 812 as indicated by arrows 815 as will be described in greaterdetail hereinbelow. From the inverse scale and offset converter 826,which performs the scale and offset step of 822 in reverse, the revertedCartesian position and velocity is input into the Cartesian controllerat 820. At the Cartesian controller 820, the original Cartesian positionand velocities as output from the FKIN controller 691 is compared withthe Cartesian position and velocity input from the simulated domain 812.If no limitations were transgressed in the simulated domain 812 thevelocity and position values input from the FKIN controller 691 would bethe same as the velocity and position values input from the simulateddomain 812. In such a case, a zero error signal is generated by theCartesian controller 820.

In the event that the desired Cartesian slave position and velocityinput at 811 would transgress one or more set limitations, the desiredvalues are restricted to stay within the bounds of the limitations.Consequently, the Cartesian velocity and position forwarded from thesimulated domain 812 to the Cartesian controller 820 would then not bethe same as the values from the FKIN controller 691. In such a case,when the values are compared by the Cartesian controller 820, an errorsignal is generated.

The type of limitations imposed on the desired slave Cartesian positionand velocity will be described in greater detail hereinbelow.

Assuming that a zero error is generated at the Cartesian controller 820no signal is passed from the Cartesian controller or converter 820. Inthe case that an error signal is generated the signal is passed througha summation junction 827 to a master transpose kinematics controller828.

The error signal is typically used to calculate a Cartesian force. TheCartesian force is typically calculated, by way of example, inaccordance with the following formula:

F_(CART)=K(Δx)+B(Δ{dot over (x)})

where K is a spring constant, B is a damping constant, Δ{dot over (x)}is the difference between the Cartesian velocity inputs to the Cartesiancontroller 820 and Δx is the difference between the Cartesian positioninputs to the Cartesian controller 820. It will be appreciated that foran orientational error, a corresponding torque in Cartesian space isdetermined in accordance with conventional methods.

The Cartesian force corresponds to an amount by which the desired slaveposition and/or velocity extends beyond the limitations imposed in thesimulated domain 812. The Cartesian force, which could result from avelocity limitation, a positional limitation, and/or a singularitylimitation, as described in greater detail below, is then converted intoa corresponding torque signal by means of the master transposekinematics controller 828 which typically includes a processor employinga Jacobian Transpose (J^(T)) matrix and kinematic relationships toconvert the Cartesian force to a corresponding torque in joint space.The torque thus determined is then input to a processor at 830 wherebyappropriate electrical currents to the motors associated with the master700 are computed and supplied to the motors. These torques are thenapplied on the motors operatively associated with the master control700. The effect of this is that the surgeon experiences a resistance onthe master control to either move it at the rate at which he or she isurging the master control to move, or to move it into the position intowhich he or she is urging the master control to move. The resistance tomovement on the master control is due to the torque on the motorsoperatively associated therewith. Accordingly, the higher the forceapplied on the master control to urge the master control to move to aposition beyond the imposed limitation, the higher the magnitude of theerror signal and the higher an opposing torque on the motors resistingdisplacement of the master control in the direction of that force.Similarly, the higher the velocity imposed on the master beyond thevelocity limitation, the higher the error signal and the higher theopposing torque on the motors associated with the master.

The imposition of the limitations in the simulated domain 812 will nowbe described in greater detail with reference to FIG. 15 of thedrawings. In FIG. 15, like reference numerals are used to designatesimilar parts or aspects, unless otherwise stated.

The slave desired Cartesian velocity is passed from the scale and offsetconverter 822 through a summation junction at 832. It will beappreciated that the slave desired Cartesian velocity is passed throughthe summation junction 832 sequentially at the rate of the controlsystem processing cycle, namely 1300 Hz. At the junction 832, an errorsignal is imparted on the slave desired Cartesian velocity when thedesired velocity of a prior desired Cartesian velocity signal would haveinstructed the simulated slave to transgress one or more limitations.This will be described in greater detail hereinbelow. If the priordesired slave velocity would not have caused a transgression, no errorsignal would have been generated and the desired slave velocity wouldthen pass through the summation junction 832 unchanged. The velocitysignal passed from the summation junction 832 is referred to asCartesian reference velocity as indicated by arrow 833.

From the summation junction 832, the Cartesian reference velocity is fedto a simulation block 834. The reference velocity is then compared withthe limitations in the simulation block 834, as will be described ingreater detail hereinbelow with reference to FIGS. 16 to 21 of thedrawings.

In the case where the slave reference velocity does not transgress alimitation, the slave reference velocity passes through the simulationblock 834 unchanged. However, a corresponding simulated slave jointvelocity is computed in the simulation block 834.

The simulated joint velocity is integrated in the simulation block 834to yield a corresponding simulated joint position. The simulated jointvelocity and position is output from the simulation block 834 asindicated by arrows 835.

The simulated joint velocity and position is then passed through afilter at 838. The filter 838 is arranged to separate tremors from thevelocity and position signals. It will be appreciated that such tremorscould result from inadvertent shaking of the master control which can beinduced on the master control by the surgeon. Since it would bedesirable to remove such tremor movements from the actual slave velocityand position signals so as to enhance slave precisional movement inresponse to master input, these tremors are filtered from the velocityand position signals by means of the filter 838. After the filteringstep at 838, resultant slave joint velocity and position signals arepassed to the slave as indicated by arrows 815 and as will be describedin greater detail hereinbelow. It will be appreciated that the simulatedslave joint position and/or velocity signal can be modified in anyrequired manner at 838. Typically, modifications not requiring feedbackto the master can be implemented at 838. Thus, the filtering step 838 isnot necessarily limited to filtering tremors from the signal only. Inaddition, or instead, the frequency of the position and/or velocitysignals may be modified to inhibit resonance in the slave, for example.

Still referring to FIG. 15 of the drawings, the simulated joint velocityand position, after passing through the simulation block 834, is routedthrough an FKIN controller at 653 to compute corresponding velocitiesand positions in Cartesian space, as described with reference to FIG. 12of the drawings. The signals are then passed to the Cartesian controller820 as already described with reference to FIG. 14.

Still referring to FIG. 15, the position signal from the FKIN controller653 is routed into a Cartesian scaled error block at 844. The desiredCartesian slave position derived from the scale and offset block 822 isalso routed into the Cartesian scaled error block 844. The two signalsare compared at 844 to compute an error signal should they notcorrespond. Should the two signals be equal, namely where the desiredslave velocity signal was not restricted in the simulated domain 834, noerror signal is generated.

In the case where the desired slave velocity was restricted in thesimulation block 834, the simulated joint velocity output would notcorrespond with the reference Cartesian slave velocity input to thesimulation block 834. Accordingly, after integration in the simulationblock 834, and conversion to Cartesian space by the FKIN controller 653,the resultant corresponding Cartesian position would not correspond withthe original desired Cartesian slave position input to the Cartesianscaled error block 844. Accordingly, an error signal of a magnitudedetermined typically by subtraction of the resultant Cartesian positionfrom the original desired position and multiplication with anappropriate constant, is generated by the Cartesian scaled error block844. This error signal is imposed on the next desired slave velocitysignal at the summation junction 832.

It will be appreciated that only the velocity signal is input to thesimulation block 834. Thus, limitations are imposed in a dynamic fashionin the simulation block. The simulated slave position does notnecessarily track the master position simultaneously. This isparticularly the case where a limitation has been imposed in thesimulation block 834. For example, should a velocity limit have beenimposed where the master was moved too quickly, a degree of lagging ofthe simulated slave position to catch up with the master positionresults. Accordingly, a discrepancy between the master and the slavepositions ensues. By means of the positional error generated at 844, anappropriate velocity signal change is effected at the junction 852 toeffect a positional “catch up” function on the velocity signal. Thus,should the master be brought to rest where a positional error isgenerated, the velocity signal input to 832 would be zero, but aCartesian reference velocity would still be input to the simulationblock 834 to effect the catching up of the simulated slave position withthat of the master.

Referring once again to FIG. 14 of the drawings, the resultant slavejoint velocity and position signal is passed from the simulated domain812 to a joint controller 848. At the joint controller 848, theresultant joint velocity and position signal is compared with thecurrent joint position and velocity. The current joint position andvelocity is derived through the sensors on the slave as indicated at 849after having been processed at an input processor 851 to yield slavecurrent position and velocity in joint space.

The joint controller 848 computes the torques required on the slavemotors to cause the slave to follow the resultant joint position andvelocity signal taking its current joint position and velocity intoaccount. The joint torques so determined are then routed to a feedbackprocessor at 852 and to an output processor at 854.

The joint torques are typically computed, by way of example, by means ofthe following formula:

T=K(Δθ)+B(Δ{dot over (θ)})

where K is a spring constant, B is a damping constant, Δ{dot over (θ)}is the difference between the joint velocity inputs to the jointcontroller 851, and Δθ is the difference between the joint positioninputs to the joint controller 851.

The output processor 854 determines the electrical currents to besupplied to the motors associated with the slave to yield the commandedtorques and causes the currents to be supplied to the motors asindicated by arrow 855.

From the feedback processor 852 force feedback is supplied to themaster. As mentioned earlier, force feedback is provided on the master700 whenever a limitation is induced in the simulated domain 812.Through the feedback processor 852 force feedback is provided directlyfrom the slave 798, in other words, not through a virtual or simulateddomain but through direct slave movement. This will be described ingreater detail hereinbelow.

As mentioned earlier, the slave indicated at 798 is provided with aplurality of sensors. These sensors are typically operatively connectedto pivotal joints on the robotic arm 10 and on the instrument 14.

These sensors are operatively linked to the processor at 851. It will beappreciated that these sensors determine current slave position. Shouldthe slave 798 be subjected to an external force great enough to inducereactive movement on the slave 798, the sensors will naturally detectsuch movement. Such an external force could originate from a variety ofsources such as when the robotic arm 10 is accidentally knocked, orknocks into the other robotic arm 10 or the endoscope arm 302, or thelike. As mentioned, the joint controller 848 computes torques requiredto cause the slave 798 to follow the master 700. An external force onthe slave 798 which causes its current position to vary also causes therequired slave movement to follow the master to vary. Thus a compoundedjoint torque is generated by the joint controller 848, which torqueincludes the torque required to move the slave to follow the master andthe torque required to compensate for the reactive motion induced on theslave by the external force. The torque generated by the jointcontroller 848 is routed to the feedback processor at 852, as alreadymentioned. The feedback processor 852 analyzes the torque signal fromthe joint controller 848 and accentuates that part of the torque signalresulting from the extraneous force on the slave 798. The part of thetorque signal accentuated can be chosen depending on requirements. Inthis case, only the part of the torque signal relating to the roboticarm 12, 12, 302 joints are accentuated. The torque signal, after havingbeen processed in this way is routed to a kinematic mapping block 860from which a corresponding Cartesian force is determined. At thekinematic block 860, the information determining slave fulcrum positionrelative to the camera frame is input from 647 as indicated. In thisregard refer to FIG. 12 of the drawings. Thus, the Cartesian force isreadily determined relative to the camera frame. This Cartesian force isthen passed through a gain step at 862 appropriately to vary themagnitude of the Cartesian force. The resultant force in Cartesian spaceis then passed to the summation junction at 827 and is then communicatedto the master control 700 as described earlier.

Reference numeral 866 generally indicates another direct force feedbackpath of the control system 810, whereby direct force feedback issupplied to the master control 700. The path 866 includes one or moresensors which are not necessarily operatively connected to slave joints.These sensors can typically be in the form of force or pressure sensorsappropriately positioned on the surgical instrument 14, typically on theend effector 58. Thus, should the end effector 58 contact an extraneousbody, such as body tissue at the surgical site, it generates acorresponding signal proportionate to the force of contact. This signalis processed by a processor at 868 to yield a corresponding torque. Thistorque is passed to a kinematic mapping block 864, together withinformation from 647 to yield a corresponding Cartesian force relativeto the camera frame. From 864, the resultant force is passed through again block at 870 and then forwarded to the summation junction 827.Feedback is imparted on the master control 700 by means of torquesupplied to the motors operatively associated with the master control700 as described earlier. It will be appreciated that this can beachieved by means of any appropriate sensors such as current sensors,pressure sensors, accelerometers, proximity detecting sensors, or thelike.

As mentioned, the control system 810 enables limitations to be set inthe simulation block 834. These limitations can be chosen to conformwith mechanical system limitations or constraints and/or can be presetto correspond with environmentally-sensitive movement limitations at thesurgical site as will be described in greater detail hereinbelow. Thus,the limitations imposed in the simulated domain 812, in one instance,can be regarded as virtual limitations corresponding with actualphysical system limitations. The limitations at the simulated domain 812are not derived from actual slave movement but from simulated or virtualslave movement. Thus, the slave is prevented from actually transgressinga limitation by simulating its movement and velocity and restricting thesimulated movement and velocity before instructing the actual slave torespond. One typical limitation set in the simulated domain 812 concernssingularities of the system.

What is meant by the term singularity will now be described by way of anexample of a singularity in the mechanical structure of the minimallyinvasive surgical apparatus. Referring to FIG. 2A of the drawings, andas already mentioned, the instrument 14 when mounted on the robotic arm10 is linearly displaceable in the direction of arrow P. If theinstrument 14 is positioned such that the end effector 58 is relativelyfar removed from the fulcrum 49 and the master control is manipulated tocommand responsive movements, the responsive movement of the slave cannormally readily be performed. At a specific fixed distance from thefulcrum 49, the end effector has a range of lateral movement constrainedwithin bounds dictated by constraints in the mechanical structure of thearm 12. It will be appreciated that the closer the end effector 58 isdisplaced toward the fulcrum 49, the smaller the possible range oflateral movement becomes. This can be visualized by picturing a conehaving its apex at the fulcrum 49 and extending from the fulcrum 49 in adownward direction in FIG. 2A. The range of lateral movement of the endeffector 58 being limited to within the visualized cone. It will thus beappreciated that toward the base of the visualized cone, e.g., a 1-inchlateral movement of the end effector, can normally readily be achievedby the mechanical structure of the arm 12. However, toward the apex ofthe cone, in other words toward the fulcrum 49, a point is reached wherea 1-inch lateral movement of the end effector 58 is simply notachievable due to the mechanical constraints of arm 12. Furthermore, themovement by the robotic arm 12 to induce lateral movement of the endeffector 58 becomes more radical the closer the end effector 58 isdisplaced toward the fulcrum 49.

When a surgeon is performing a surgical procedure by means of theminimally invasive surgical apparatus, he or she is normally unaware ofthe robotic arm 12 movements since he or she is viewing the surgicalsite through the viewer 202. Accordingly, unless provision is made tothe contrary, it could happen that in the course of a surgical procedurethe end effector 58 is displaced too near the fulcrum 49 so that masterinput causes the robotic arm 12 to move too quickly over correspondinglong distances in responding to the commanded end effector movements.The control system 810 is arranged to provide a method of inhibiting therobotic arm from making too rapid or large a movement in response tomaster input because of the singularity described above.

Another singularity of the mechanical structure of the slave, inparticular of the surgical instrument 14, will now be described withreference to FIG. 5 of the drawings.

As mentioned, the end effector 58 is angularly displaceable about axis14.2 as indicated by arrows 59. Should the axis of symmetry 60A of theend effector be positioned along the axis 14.2, angular displacement ofthe end effector about axis 60A is readily induced. However, should theaxis 60A be positioned perpendicular to the axis 14.2, angulardisplacement of the end effector 58 about axis 60A is not possible.Thus, a singularity is approached as the axis 60A approaches a positionperpendicular to the axis 14.2.

A further singularity of the robotic arm 10, can be understood withreference to FIG. 4 of the drawings. As already mentioned, the roboticarm is angularly displaceable about axis 28 as indicated by arrows 26.When the axis 14.2 is perpendicular to the axis 28, movement of the arm10 in the direction of arrows 26 is readily induced on the end effector58. As will readily be observed in FIG. 4, a singularity is approachedthe closer the axis 14.2 is moved toward a position parallel to the axis28.

Another typical limitation imposed in the simulated domain 812 relatesto positional constraints of the various joints.

Another typical limitation imposed in the simulated domain is a velocitylimitation corresponding to practicably mechanically achievable slavevelocity. Naturally, the slave has greater mass and moments of inertiathan the master. Thus, should the surgeon move the master too quickly,or should the master accidentally be knocked to induce rapid movementthereon, the slave would be commanded to move in sympathy with themaster but at a rate not practicably achievable by the arm 10 due tomechanical constraints.

As mentioned, optionally, limitations relating to surgical environmentalconstraints can also be effected as described in greater detailhereinbelow.

Referring now to FIG. 16 of the drawings, one embodiment of thesimulation block 834 includes a modified Jacobian inverse controllerindicated by J^(−1*) at 870. The modified Jacobian inverse controller isarranged to inhibit the detrimental effects which result when asingularity is approached. This is achieved by modifying a Jacobianinverse matrix of the controller J^(−1*). The modification to the matrixwill now be described by way of example and with reference to FIG. 2Aand 17 of the drawings.

In FIGS. 2A and 17, the length of the arm portion of the shaft 14.1 ofthe instrument 14 which extends beyond the fulcrum 49 is indicated by L.

The relationship between velocity {dot over (x)} in Cartesian spacerelative to angular velocity {dot over (θ)} in joint space is typicallyexpressed by the relationship

{dot over (x)}=J.{dot over (θ)}

For the minimally invasive surgical apparatus, the Jacobian matrix istypically in the form of a 6×6 term matrix for converting joint spacecoordinates to corresponding Cartesian coordinates. Naturally, some ofthe terms in the matrix include a multiplication factor equal to L.Accordingly, when it is required to determine positions in joint spacecorresponding to Cartesian coordinates, the following relationship isused:

{dot over (θ)}=J⁻¹.{dot over (x)}

When the inverse Jacobian matrix is used in this fashion, the termsincluding the multiplication factor of L become terms having amultiplication factor of 1/L.

It will be appreciated that as L decreases the term 1/L approachesinfinity. This characteristic associated with a singularity isschematically illustrated in FIG. 17. The length L is indicated alongthe horizontally extending axis and the corresponding factor 1/L isindicated along the vertically extending axis. The parabolic linesindicate the relationship between L and 1/L. It is clear that when therequired joint velocity is determined by means of the Cartesian velocity{dot over (x)} and a term includes the multiplication factor 1/L, thejoint velocity approaches infinity as the value of L decreases, thus asthe end effector is moved closer to the fulcrum 49.

To compensate for these detrimental effects when a singularity isapproached, the 1/L term in the Jacobian Inverse matrix is replaced witha function of L which yields a resultant relationship between L and 1/Las indicated in dashed lines in FIG. 17. Two dashed lines are indicatedto show different possible functions of L. In similar fashion theJacobian Inverse matrix is modified to cater for all the singularitiesof the system already described.

Referring again to FIG. 16 of the drawings, the simulation block 834will now be described in further detail.

The modified Jacobian Inverse controller which makes allowance forsingularities as hereinbefore described is indicated by the referencenumeral 870. The Cartesian space reference velocity is input asindicated by arrow 833. After conversion to a resulting joint velocityby the controller 870, the resultant joint velocity is output at 874.The resultant joint velocity 874 is then input to a joint velocitylimitation step at 876. At this step the resultant joint velocity islimited to remain within a range between a predetermined maximumvelocity V_(max), and a predetermined minimum velocity V_(min). Thesemaximum and minimum values are typically selected to constrain the jointvelocity within limitations corresponding to constraints of themechanical structure of the system. Accordingly, at 876, should thejoint velocity input 874 have a magnitude greater than the maximum andminimum values, the joint velocity magnitude 874 is decreased to withinthe set range. Thus:

if {dot over (θ)}>max {dot over (θ)}=max

if {dot over (θ)}<(min) {dot over (θ)}=(min)

where {dot over (θ)} represents joint velocity, and max denotes apositive magnitude and min denotes a negative magnitude.

After the joint velocity is limited in this manner, the joint velocityis integrated at 878 to yield a corresponding position in joint space.In similar fashion to the joint velocity limitation step at 876, theposition is limited at 880 to remain within a set positional range.

From 880, the resultant joint positional signal is routed to the filter838 as indicated by one of the arrows 835 and as already describedherein with reference to FIG. 14. The resultant velocity signal asoutput from 876 is routed to the filter 838 as indicated by the otherarrow 835. The resultant velocity signal is linked to the positionalcontrol step 880, as indicated at 881, so that in the event that theposition is limited, the velocity signal is rendered zero.

As mentioned, velocity, position and singularity limitations orconstraints are applied to the Cartesian reference velocity in thesimulation block 834 indicated in FIG. 14 to yield a simulated slavejoint position and velocity. Naturally, should the Cartesian referencevelocity input to the simulation block 834 not result in a transgressionof any of the limitations set for the slave, the Cartesian referencevelocity input to the simulation block 834 is then simply transferredinto corresponding slave joint position and velocity signals without anyimposition of limitations. The corresponding slave joint position andvelocity is then forwarded to the slave after the filtering step at 838.

An alternative simulation block 834B and another method of imposinglimitations will now be described with reference to FIG. 18 of thedrawings in which like reference numerals are used to designate similarparts unless otherwise indicated.

Referring now to FIG. 18, and in the simulation block 834B, theCartesian reference velocity is initially input into a Cartesianposition and velocity limit block at 902. At 902, any desiredlimitations to position and velocity in Cartesian space can be set. Thiscan be achieved in similar fashion to the manner in which the jointvelocity and position limitations were imposed in FIG. 16. Suchlimitations can be chosen to suit the specific surgical procedure to beperformed. Thus, for example, should the surgical procedure to beperformed be at a sensitive location, such as close to the brain, orheart, or the like, limitations can be set to constrain end effectormovement within a space so as not to be able to contact the area ofsensitivity. Thus, at 902, limitations can be tailored to meet specificenvironmental limitations defined by the specific surgical procedure tobe performed so as to avoid accidental damage to a sensitive organ, orthe like. Thus, at 902, slave position and velocity can be restricted toremain within preset limitations dictated by the surgical procedure tobe performed. It will be appreciated that such surgical environmentdependent limitations can be imposed in the simulation block 834 in FIG.16, and also in the preferred simulation block 834A to be discussed withreference to FIG. 20.

After the limitation step at 902, the resultant Cartesian velocity isinput to a modified Jacobian Inverse controller at 904. The modifiedcontroller 904 imposes limitations on the Cartesian velocity inputduring conversion of the Cartesian velocity input into a correspondingjoint space velocity to make allowance for singularities as alreadydescribed.

From the modified Jacobian Inverse controller 904, the resultant jointvelocity is input into a joint position and velocity block at 906. Atthe joint position and velocity block 906, the joint velocity input ismonitored to ensure that corresponding velocity and position commands toeach specific joint would not transgress set limitations correspondingto actual angular position and velocity limitations of that joint. Afterthe joint velocity has been monitored at 906, and any limitationsimposed, the resultant simulated slave joint velocity is output asindicated by arrow 835. The simulated slave joint velocity is also fedthrough an integration step at 910 to yield the corresponding simulatedslave joint position.

The simulated joint position for each specific joint is routed to thejoint position and velocity block 906, and the modified Jacobian Inverseblock 904 as indicated in dashed lines. The position signal 835 isrouted to the modified Jacobian Inverse block 904 to enabletransformation from Cartesian to joint space. The position signal 835 isrouted to the position and velocity block 906 in order that jointposition and velocity limits can be imposed at 906. This will now bedescribed with reference to FIG. 19 in which like reference numerals areused to designate similar parts unless otherwise indicated. It will beappreciated that FIG. 19 exemplifies the imposition of positional andvelocity limits on a single joint. The same method of imposing suchpositional and velocity limits is employed for each joint at 906.

In FIG. 19, the joint velocity input from the modified Jacobian Inversecontroller at 904 is indicated by arrow 912. The resultant velocityafter having passed through the joint position and velocity block isindicated by arrow 914 and the joint position input is indicated byarrow 835 and is shown in dashed lines. The joint for which position andvelocity limits are to be imposed by the block diagram shown in FIG. 19normally has physical limitations. Thus, the joint has a maximumposition in which the arm members which are pivotally connected therebyare at a maximum angular position relative to each other. Similarly, thejoint has a minimum position in which the arm members which areconnected one to another thereby are at a minimum angular positionrelative to each other. Accordingly, the joint has an angulardisplacement range extending between its minimum and its maximumposition. The angular limits of the joint are indicated by blocks 918and 920, respectively, block 918 indicating the minimum position andblock 920 the maximum position. Naturally, since we are dealing with asimulated domain, the limits can be chosen to suit. Accordingly, theminimum and maximum angular positions 918, 920 need not necessarilycorrespond with the actual physical positional limits of the joint, butcan be chosen at any appropriate angular positions within the angularpositional range capabilities of the joint.

The position input at 835 is normally varying continually as the surgeonmanipulates the master during the course of a surgical procedure. Thepositional input 835 is fed to the summation junctions 922, 924. At thejunction 922, the angular position as input at 835 is compared with thepositional minimum or lower limit to yield an angular valuecorresponding to the angular deviation of the position input 835relative to the limit 918. Thus, at 922, an angular value equal to thedifference between the angular limit and the angular position input 835is determined. The angular deviation from the lower limit 918 thusdetermined, is then fed to a velocity determination block at 926. Theprocessing cycle rate of the control system is known. In this case, itis typically 1300 Hz. At 926, the velocity which the joint needs to haveto cause its position to coincide with the lower joint limit 918 at thenext processing cycle is determined. This velocity value is then routedto a decision block at 928. Naturally, if the angular position as inputat 835 is far removed from the lower limit 918, the resultant velocityvalue derived at 926 will be very large, and typically physicallyunattainable. However, as the angular deviation approaches zero, namely,where the angular position 835 approaches the lower limit 918, thevelocity output from 926 becomes less than the attainable joint velocityand becomes zero where the angular position 835 is at the lower limit918.

Reference numeral 930 represents a set joint velocity limit. This limitis typically chosen in accordance with the acceptable joint velocitylimit of that joint. This set velocity lower limit is also fed into thedecision block 928. At 928 the two joint velocities are compared and thelargest of the two selected. It will be appreciated that the largestvalue is selected because we are regarding a velocity limit in anegative direction. Thus, the largest value is the same as the smallestabsolute value. The selected velocity value thus determined defines thelower velocity limit as indicated at 932.

It could happen that the joint is positioned beyond the positional lowerlimit 918. This can occur when the minimally invasive surgical apparatusis initially setup, or where the positional limits are selectivelychanged, for example. In such a case, it is desirable to cause the jointposition to return to within the range set by the upper and lower limitsat 918 and 920, respectively. For the lower angular position limit, thisis achieved by the block 934. In essence, what is achieved by the block934, is a constant curbing of positional movement beyond the lowerlimit. Thus, as the surgeon manipulates the master, movements causingthe angular position of the joint to move toward the limit arepermitted, but once such movement has taken place, the joint isrestricted to its new position closer to the limit. The process ismaintained until the joint position is within the range set by thevalues at 918, 920, respectively.

It will be appreciated that a maximum velocity, as indicated byreference numeral 935 is determined in similar fashion as the minimumvelocity, as can be seen in FIG. 19 of the drawings.

Referring now to FIG. 20 of the drawings, a preferred simulation block834A will now be described. In FIG. 20 the same reference numerals areused to designate similar parts or aspects unless otherwise stated.

In FIG. 20, the Cartesian reference velocity is input as indicated byarrow 833. The simulated joint positions and velocities are output at835. The Cartesian reference velocity 833 is routed to a modified fullJacobian Inverse block at 942 and to an isolation block at 944.

At 942, the Cartesian reference velocity signal 833 is transformed intoa corresponding joint velocity signal 946. The modified full JacobianInverse block 942 makes allowance for singularities as already describedwith reference to 904 in FIG. 18.

In the minimally invasive surgical apparatus under discussion, themodified full Jacobian Inverse block typically includes a six by sixterm matrix. After transformation at the block 942, the resultant jointvelocity signal is passed to an isolation block 948. At the isolationblock 948, the terms relating to the wrist joints, as indicated in FIG.5 of the drawings, are isolated from the terms relating to the joints onthe robotic arm 12, as indicated in FIG. 2A and 2B. After isolation at948, the wrist joint velocities are forwarded to a wrist joint velocityand position limitation block at 950.

At 950 wrist joint velocity limits are imposed on each wrist joint insimilar fashion to the method described above with reference to FIG. 19.However, for the wrist joints, namely the joints providing the threedegree of freedom of movements to the end effector 58, the limitationsare imposed simultaneously rather than on a joint by joint basis. Thiswill now be described with reference to FIG. 21.

Referring to FIG. 21, the limits for each joint are determined insimilar fashion to that described with reference to FIG. 19. But, asindicated at 970, the limitations are used to define a correspondingvelocity limitation for the three joints together as indicated by thebox 972. Accordingly, a multidimensional joint velocity limitation, inthis case a three-dimensional joint velocity limitation, is provided.

The input joint velocity signal at 951 is compared to themultidimensional joint velocity limitation, at 970. Should the inputvelocity signal 951 fall entirely inside the limitation, it is unchangedby the limitation. In such a case the output velocity signal 952 is thesame as the input velocity signal 951. However, should the inputvelocity signal 951 fall outside the limitation, the limitation block at970 will select the output velocity 952 according to a criterion, whichwill now be described.

A joint velocity error between the input velocity signal 951 and theselected output velocity 952 is defined as illustrated at 974. The jointvelocity error is transformed into a Cartesian velocity error using aJacobian matrix at 976. It will be appreciated that the Jacobian matrixat 976 describes the kinematics of the wrist joints, which includespivots 54, 60 and axis 14.2, with reference to FIG. 5. The magnitude ofthe Cartesian velocity error is then determined at 978.

The criterion for selection of the output velocity 952 by the limitationblock 970 is the obedience of the multidimensional limitation and theminimization of the Cartesian velocity error magnitude.

Returning now to FIG. 20 the drawings, the output 952 from thelimitation block 950 represents a combined joint velocity signalincluding joint velocities at the joints or pivots 54, 60 and jointvelocity about axis 14.2, with reference to FIG. 5 of the drawings,after any limitations relating to velocity, position and singularitieshave been imposed.

At the isolation block 944, the translational Cartesian velocity termsare isolated from the Cartesian reference velocity signal 833. Theisolated terms correspond to the Cartesian velocity commands addressingthe joints on the robotic arm 12. After isolation, the Cartesianreference velocity signal for the outer joints only is forwarded to anadjustment block at 954.

In the event that the wrist joint velocity signal was restricted at oneor both of the blocks 942, 950, the outer joint velocity can be adaptedat 954. This will now be described in greater detail and with referenceto FIG. 5 of the drawings.

It will be appreciated that a command at the master control 700 relatingto only an orientation change of the end effector 58 can result in notonly responsive angular movement about pivots 54, 60 and about axis 14.2but also responsive outer joint movement. This is so because ofstructural dissimilarities between master and slave. Thus, for the slaveto perform an orientational movement corresponding to a masterorientational movement, it is sometimes required for the slave outerjoints to move also.

Accordingly, in the event that wrist joint velocity limits were imposed,it is required to adapt outer joint, or translational, velocity to theextent to which the outer joint velocity formed part of theorientational wrist limitation. This is achieved at 954.

The resultant, possibly adapted, translational Cartesian velocity signalis then forwarded to a modified translation Jacobian Inverse block at956. At 956, the signal is converted into a corresponding joint spacevelocity signal. The modified Jacobian Inverse matrix at 956 makesallowance for the fulcrum 49 singularity and the maximum robotic armpitch singularity as already described with reference to FIG. 4. Thejoint space velocity signal from 956 is then passed to a limitationblock at 958. At 958 positional and velocity limitations are imposed onthe signal in a manner similar to that already described with referenceto FIG. 19 of the drawings, and for each outer joint.

The final wrist joint velocity signal and the final outer joint velocitysignal are then combined at 960 to yield the simulated joint velocity835. The simulated joint velocity 835 is integrated at 962 to yield acorresponding simulated joint position, indicated by the other of thearrows 835.

The simulated joint position is fed to the blocks 942, 950, 954, 956 and958 to enable the required computations.

As mentioned, during the course of a surgical procedure, the surgeon maywish to perform certain actions such as varying the endoscope positionby, e.g., causing the endoscope to displace relative to its fulcrum toview a different part of the surgical site or to permit a specific partof the surgical site to be magnified, or causing the endoscope todisplace angularly about its shaft axis angularly to displace theorientation of the surgical site as displayed on the viewer, or thelike.

It will be appreciated that, when for example, the endoscope position ischanged, the position and orientation of the end effectors 58 as viewedin the image displayed on the viewer 202 normally also changes. In sucha case, the mapping of the position and/or orientation of the pincherformation 706 on the end effectors 58 as viewed in the viewer 202 isdisturbed and normally no longer corresponds. To re-establishorientational and/or positional mapping of the pincher formations 706onto the new positions and/or orientations of the end effectors 58 asdisplayed on the viewer 202, it is required to re-map or realign pincherformation orientation and/or position with that of its associated endeffector 58 in the changed image.

It will be appreciated that such re-mapping or realigning of pincherformation 706 with associated end effector 58 is not necessarily onlyrequired after a change in the image due to changing endoscope positionor orientation. It is also required when, e.g., one of the surgicalinstruments 14 is replaced with another bearing a different end effectorrequired to perform a specific surgical task. It is also required whenthe positions of the master controls in the space 206 is changed withoutchange to end effector position. It is also required when the fulcrumpoint 49 is moved to a different incision point or port of entry, bymoving the set up joint arms 95. It is also required when the arm 12 andinstrument 14 are moved by, e.g., a surgical assistant, without matchingmotion of the pincher formation 706. Accordingly, realigning is normallyrequired when normal, or on-line, control between master and slave hasbeen interrupted in a manner causing their mapping or alignment to bedisturbed.

The control steps followed by the control system of the minimallyinvasive surgical apparatus to achieve such a re-mapping or realigningof pincher formation 706 with its associated end effector 58 will now bedescribed with initial reference to FIG. 22 and then in greater detailwith reference to the Figures that follow.

To perform an action like changing endoscope position, on-line controlbetween master and slave, as hereinbefore described, is interrupted.This can typically be achieved by activating an input such as pressing abutton, depressing a foot pedal, through voice control, or the like.This step is indicated at 502 in FIG. 22.

After on-line control between master and slave is interrupted in thismanner, the required action, as indicated at 504, such as repositioningthe endoscope, or the like, is performed off-line. Upon completion ofthis action, the pincher formation 706 is caused to realign with a newposition and/or orientation of its associated end effector 58 as viewedin the new image. Upon disconnection of on-line control between masterand slave, the slave is normally locked in its position. After theendoscope 304 position is changed, the position and orientation of theend effector 58 relative to the camera frame 610 normally also changes.Thus, to realign the pincher formation 706 with its associated endeffector 58, the pincher formation 706 is caused to move into a newposition and/or orientation relative to its reference frame, the eyeframe 612, at which its orientation and/or position is once again mappedonto the position and/or orientation of the new position and/ororientation of its associated end effector 58 relative to the cameraframe 610. This re-mapping of pincher formation position and/ororientation with that of its associated end effector 58 is referred toas master alignment in this specification and is indicated at 506 inFIG. 22.

The master alignment step 506 involves computing a new required positionand/or orientation of the pincher formation 706 relative to the eyeframe 612, at which position and/or orientation it is once again mappedonto the position and/or orientation of its associated end effector 58relative to the camera frame 610. This step is indicated at 508. Afterthe new required position and/or orientation of the pincher formation706 is determined, the pincher formation 706 is caused to move to thenew required position and/or orientation, as indicated at 510. This istypically achieved by the actuators, e.g., electric motors, associatedwith the master 700.

It will be appreciated that the movement of the pincher formation 706 tothe new required position and/or orientation as indicated at 510 neednot be achieved automatically through the use of actuators.Determination of when the pincher formation 706 reaches its new positionand/or orientation can be performed by comparing a pincher Cartesianposition and/or orientation to the required new Cartesian positionand/or orientation or by comparing the pincher joint position to therequired new joint position. Accordingly, once the new required positionand/or orientation has been determined, movement of the pincherformation to the new required position and/or orientation can beachieved manually, e.g., by the surgeon. Appropriately positioned brakesor locks can be provided on the joints of the master control 700 tocause the joints automatically to lock as each joint reaches a positionin which it is in register with a required position of that joint toestablish the new required position and/or orientation of the pincherformation. Instead, the control system can be arranged to cause a signalor cue, e.g., visual, or audio, or the like, to be presented to thesurgeon when he has moved the pincher formation into a position and/ororientation in register with the new required position and/ororientation.

After realignment of the master 700 with respect to the slave has beenperformed in this manner, the on-line control between master and slavecan be re-connected as indicated at 512. The re-established controlbetween master and slave is then performed under the new conditions asindicated at 514 in which, e.g., the endoscope position has beenchanged.

It will be appreciated that steps 506, 512 and 514 may also be used toinitially align the master 700 to the slave and to establish the initialcontrol between master and slave, after the minimally invasive surgicalsystem is turned on and an initial surgical instrument is mounted on oneof the robotic arms 12.

The master step 508 will now be described in greater detail withreference to FIG. 23.

The steps in realigning the master with the slave described hereinbeloware described by way of example with reference to a required actionwhere, e.g., the endoscope has been moved into a new position. Where therequired action is for, e.g., a surgical instrument change, the stepsare similar.

In the case of a surgical instrument change, the surgical instrumentmounted on one of the robotic arms 12 is replaced with another surgicalinstrument. When the new surgical instrument is mounted on the roboticarm 12, it is introduced into the surgical site such that its endeffector orientation is at a specific orientation relative to theworking end of the instrument. Conveniently, the orientation is suchthat the axis of symmetry 60A of the end effector is parallel with therotational axis of the shaft 14.

Since end effector orientation during surgical instrument change isnormally the same irrespective of the specific surgical instrument to bemounted on the robotic arm 12, the orientation and position relative tothe fulcrum frame 630 is known before the surgical instrument isactually replaced. Thus, the master can be caused to move into ananticipated aligned condition during the performance of a surgicalinstrument change operation, before or while the new surgical instrumentis actually being mounted on the robotic arm 12.

Returning now to FIG. 23 of the drawings, and as already mentioned, theslaves are locked in position upon disconnection or interruption ofon-line control. Once the required off-line action has been performed,the fulcrum frame 630 of the slave 798 relative to the camera frame 610,and the slave joint positions (of the robotic arm 12 and surgicalinstrument 14) are input to an FKIN controller at 516. The FKINcontroller 516 determines end effector frame 618 position andorientation in Cartesian space with respect to the camera referenceframe 610. The Cartesian position and orientation of the end effectorframe 618 relative to the camera frame 610 is then forwarded to aninverse scale and offset block at 518, as indicated by arrow 520. Thecorresponding position and orientation of the associated pincherformation 706 relative to the eye frame 612 is then determined. Thiscorresponding position and orientation of the pincher formation 706relative to the eye frame 612 represents the position and/or orientationinto which the pincher formation 706 is to be moved so that its positionand/or orientation in the eye frame 612 is again mapped onto the newposition and/or orientation of the end effector frame 618 relative tothe camera frame 610.

The corresponding position and/or orientation of the pincher formation706 relative to the eye frame 610 is then input to a selection block at522, as indicated by arrow 524. At 522 a selection is made as to whetheronly the position of the master is to be aligned with the slaveposition, only the orientation of the master is to be aligned with theslave orientation, or both.

In the present system, the selection involves aligning or mapping masterorientation with slave orientation only, for practical reasons. However,the option of aligning master position with slave position is alsoavailable. In the description which follows, the steps involved inaligning or mapping master orientation and position with slaveorientation and position will be described.

After the selection at 522, a desired master Cartesian orientation andposition relative to the eye frame 612 is then output and forwarded toan isolation block at 526 as indicated by arrow 528.

At 526, the desired master Cartesian position and orientation withrespect to the eye frame 612 is compared with the platform frame 640position relative to the eye frame 612, to compute the desired masterCartesian position and orientation relative to the platform frame 640.The desired master Cartesian position and orientation relative to theplatform frame 640 is then output from 526 as indicated by arrow 530.

The desired master Cartesian position and orientation with respect tothe platform frame 640 is then input to a master inverse kinematicsblock at 532.

In the master inverse kinematics block 532, the master outer jointpositions are determined at 534. These outer joint positions are thepositions of the joints on the articulated arm portion 712 of the mastercontrol 700 as can best be seen in FIGS. 6B and 6C.

Thereafter, the master wrist joint positions are determined at 536. Themaster wrist joints are the joints on the wrist gimbal or hand held part699 as can best be seen in FIGS. 6A and 6C. To determine the masterwrist joint positions at 536, information relating to the slave jointpositions are required as indicated by arrow 538 and as will bedescribed in greater detail hereinbelow. The desired master jointposition, which defines pincher formation 706 position and orientation,is then output as indicated by arrow 560.

The operation of the master inverse kinematics block 532 will now bedescribed in greater detail with reference to FIG. 24.

The desired master Cartesian position and orientation with respect tothe platform frame 640 is split at 540 to yield the Cartesian positionof the master relative to the platform frame 640, namely the position ofthe master frame 622 having its origin at 3A relative to the platformframe 640, and the Cartesian orientation of the master relative to theplatform frame 640, namely the orientation of the master frame 622relative to the platform frame 640.

If the origin of the master frame 622 is offset from the intersection3A, as discussed hereinabove, this offset is used at 542 to determinethe point of intersection 3A position relative to the platform frame 640as indicated by arrow 544. To enable this, the Cartesian orientation ofthe master frame 622 relative to the platform frame 640 is required asindicated by arrow 543.

The intersection 3A position relative to the platform frame 640 is theninput to an inverse kinematics block at 546 to yield master outer jointpositions, as indicated at 548.

The Cartesian orientation of the master frame 622 relative to theplatform frame 640 as indicated by arrow 543 and the master outer jointpositions as indicated by arrow 548 are forwarded to an adjustment blockat 550. It will be appreciated that the orientation of the pincherformation 706 will be affected if the articulated arm 712 position ismoved operatively upwardly or downwardly, as can best be seen in FIG. 6Cand as indicated by arrows G. Thus, at 550, the desired Cartesian masterorientation relative to the platform frame 640 is adjusted to accountfor any affect which master outer joint position could have had onmaster orientation. Accordingly, after the step at 550, the desiredpincher formation 706 orientation relative to the free end of thearticulated arm portion 712 on which the wrist gimbal 699 is mounted, isdetermined, as indicated by arrow 552.

The desired pincher formation 706 orientation relative to the free endof the articulated arm 712 is then forwarded to a selection block at554. At 554 it is decided whether the orientational realignment of themaster corresponds to a major, or absolute, realignment operation or aminor, or delta, realignment operation. This decision depends on thetype of off-line action which was required initially, and which wasexecuted at 504. Actions which can give rise to major misaligned betweenmaster and slave are typically actions like, surgical instrument change,change of port of entry (set up arm 95 position change), at systeminitialization, when operative association between master and slave isswapped, and the like. Actions which typically give rise to minormisaligned are for instance when the surgeon during the course of asurgical procedure repositions the master control 700 at the surgeon'sconsole 200, whilst its associated slave is locked in position, duringendoscope position change, and the like.

When the misalignment can be major, an absolute master wrist alignmentprocedure is performed, as indicated at 556. When the misalignment wouldbe minor, an incremental or delta master wrist alignment procedure isperformed, as indicated at 558. After the required alignment procedure,as indicated at 556 or 558, the wrist gimbal 699 joint positions aredetermined and are combined with the master outer joint positions toyield desired master joint positions, as indicated at 560.

The absolute master wrist alignment procedure will now be described ingreater detail with reference to FIG. 25.

The desired orientation of the master frame 622 relative to the free endof the articulated arm portion 712 is input to a solution block at 562.

The configuration of the master wrist gimbal 699 is such that aplurality, in this case an infinite number, of solutions relating to theposition of the wrist gimbal links to yield the desired orientation ofthe master frame 622 relative to the free end of the articulated armportion 712, are possible. This can best be explained with reference toFIGS. 28 to 30. In FIGS. 28 to 30, three different configurations of thewrist gimbal 699 are shown which provide the same pincher formation 706orientation, relative to the free end position of the articulated armportion 712 positioned as indicated generally at 702C. With particularreference to FIGS. 29 and 30, it will be seen that an infinite number ofsolutions, as indicated by arrows T, are provided since the arm members702A and 702D can occupy any position within the associated joint rangesat 704A and 704D about the axes 4 and 2 to yield the same pincherformation orientation. Furthermore, the wrist gimbal 699 can be movedinto a configuration indicated in FIG. 28 to yield the same pincherformation orientation. In the configuration shown in FIG. 28, anotherset of infinite solutions, as indicated by arrows R, is provided sinceonce again the arm members 702A and 702D can occupy any position withinthe joint ranges of the joints 704A and 704D about the axes 4, 2 toyield the same pincher formation orientation.

The infinite solutions corresponding to the wrist gimbal configurationindicated in FIGS. 29 and 30 define one segment of infinite solutions,and the solutions corresponding to the wrist gimbal configurationindicated in FIG. 28 define another segment of infinite solutions.

Returning now to FIG. 25 of the drawings, these segments are defined at562 and can vary depending on the mechanical structure of the wristgimbal 699.

To determine a preferred wrist gimbal configuration and arm 702 angularpositions about axes 1, 2, 3 and 4, each segment is passed through anoptimization solution block 564.

For the configuration indicated in FIGS. 29 and 30, an arbitraryposition or angular orientation of the arm member 702D at joint 704D isselected from the segment of infinite solutions at 566. Based on thisinitial arbitrary position an iterative process is followed three timesto approach an optimum position of the arm members 702, as indicated at568. The iterative process involves forwarding the initial arbitraryposition of the arm 702D to an inverse kinematics block at 570. Thepincher formation orientation relative to the free end of thearticulated arm 712 is input to the inverse kinematics block asindicated by arrow 552. The joint positions of all the joints on thewrist gimbal 699 are determined based on the initial arbitrary positionof the arm 702D. These joint values are then passed through a costfunction block at 572. Based on the value determined by the costfunction, a new position of the arm 702D is determined which shouldyield a more favorable cost function value. The process is repeatedthree times as mentioned and the resultant position of the arms 702together with their cost function value are forwarded to a selectionblock at 574. This process is repeated for each solution segment.Thereafter, the configuration and arm 702 positions corresponding to thelowest cost function value is selected at 574 to yield the jointpositions of the wrist gimbal 699, as indicated by arrow 576. It will beappreciated that the cost function can be selected to regulate theoptimization process. In particular, the cost function may evaluateangular positions of joints 704 relative to a center of their respectiveangular displacement ranges, thereby to approach an optimization of thewrist gimbal 699 range of motion.

The pincher formation 706 is rotatably mounted on the wrist gimbal 699at the joint 704B. The joint 704B is arranged to permit the pincherformation 706 to displace angularly through multiples of 180°. It willbe appreciated that there are accordingly multiple solutions about theaxis 1 at joint 704B in which the pincher formation 706 is mapped ontothe fingers of its associated end effector 58. During the absolutealignment procedure 556, a preferred angular position of the pincherformation 706 within its angular displacement range about axis 1 isdetermined. Determining the preferred angular position of the pincherformation 706 will now be described in further detail with reference toFIGS. 25 and 26.

The pincher formation orientation relative to the free end of thearticulated arm 712 is compared with the slave joint position at 578. At578 the effect of causing the slave joint positions, at 54 and 60, tochange, so as to cause the end effector 58 to be in a position in whichits axis of symmetry 60A is parallel with the axis of the shaft 14.1, issuperimposed on the pincher formation orientation relative to the freeend of the articulated arm 712. This yields a simulated pincherformation orientation or angular position relative to the free end ofthe articulated arm 712 which corresponds with end effector angularorientation if moved into a position in which axis 60A is parallel withthe axis 14.2. This simulated orientation of the pincher formation 706is referred to as a reference orientation, as indicated by arrow 580.

The reference orientation is then forwarded to an inverse kinematicsblock at 582. In the block 582, joint positions corresponding to theCartesian orientation of the pincher formation 706 are determinedconveniently defining the axes 1 and 3 of the wrist gimbal as beingperpendicular.

A reference simulated wrist joint solution for the wrist gimbal 699 isthus determined, as indicated by arrow 584. The simulated wrist jointsolution is then forwarded to a solution block at 586. At 586 the slaveangular position with respect to its roll axis 14.2 within its angulardisplacement range about 14.2 is input to 586, as indicated by arrow588.

At 586, the angular position of the slave within its angulardisplacement range about its roll axis 14.2 is compared with the angularposition of the pincher formation within its angular displacement rangeabout axis 1 of the reference simulated wrist joint solution. In FIG.26, the angular position of the slave about 14.2 within its angulardisplacement range 555A about 14.2 is indicated at 551 by way ofexample, its angular range about 14.2 extending between a minimum,indicated at 553, and a maximum, indicated at 555. The angulardisplacement range of the pincher formation 706 about axis 1 isindicated by reference numeral 559A as extending between a minimum,indicated at 557, and a maximum indicated at 559. The angular positionof the pincher formation 706 about axis 1 as input from 582 (refer toFIG. 25) is indicated at 561, for example. The angular position of thepincher formation as indicated at 561 is derived from the referencesimulated wrist joint solution.

It will be appreciated that, since the pincher formation 706 can bedisplaced through multiples of 180°, a plurality of angular positionsexist which correspond to the angular position at 561. These angularpositions are indicated at 563. At 586 in FIG. 25, the angular position551 of the slave about its roll axis 14.2 within its angulardisplacement range is compared with the various angular positions 561,563 within the angular range of the pincher formation 706 about axis 1.The angular position 561, 563 relative to the range between 557, 559which best matches the angular position 551 relative to the rangebetween 553, 555 is then selected as indicated at 563A.

As mentioned with reference to FIG. 5, the end effector 58 has twomembers or fingers 58.1, 58.2, which will be referred to as “rightfinger” and “left finger.” Similarly, with reference to FIG. 6A, thepincher formation 706 has two opposing members, upon which the surgeonplaces his thumb and index finger, which will be referred to as “rightpincher” and “left pincher.” It will be appreciated that the left andright fingers of the end effector 58, as well as the left and rightpinchers of the pincher formation 706 may appear symmetric and therebyindistinguishable to a user, but that the control system 810 candistinguish between them.

It will be appreciated that there are two relationships in which thepincher formation 706 can be mapped to its associated end effector 58,namely, either relating the right pincher to the right finger, orrelating the right pincher to the left finger. Displacing the angularposition of the pincher formation 706 by 180° about axis 1 will toggleits mapping to the end effector 58 between these two relationships.

By construction, the angular position of the pincher formation 706 aboutaxis 1 as input from 582 and as indicated at 561 places the pincherformation in the reference orientation and relates the right pincher tothe right finger. Accordingly, a 180° angular displacement from 561would yield a right pincher to left finger relationship. As can be seenin FIG. 26, the comparison in the present example yields a relationshipof right pincher to left finger at 563A. It will be appreciated thatthis step is performed merely to determine an appropriate branch,namely, either relating right pincher to right finger or relating rightpincher to left finger.

Returning now to FIG. 25 of the drawings, when the branch has beendetermined in this manner, the relationship is forwarded to a selectionblock at 590, as indicated by arrow 592 in FIG. 25. The referencesolution value, at 563A, is also forwarded to the selection block 590 asindicated by arrow 594.

At 590 and referring to FIG. 26, the angular position 565 of the pincherformation 706 relative to its range, as derived from 576, is indicatedby reference numeral 565A. Once again, a plurality of angular positionsexist which correspond to the angular position at 565, as indicated byreference numerals 567. By construction, the angular position 565,indicated at 565A, relates the right pincher to the right finger. Inorder to determine an appropriate solution, only those angular positionsmatching the branch already determined at 586, namely, in the presentexample, relating right pincher to left finger, are considered asappropriate solutions.

Accordingly, the right pincher to left finger angular position whichbest matches the angular position at 563A is selected as indicated byreference numeral 567A. This angular position value is then selected asthe angular position to which the pincher formation 706 is to bedisplaced to align with the end effector. This angular position,together with the other joint positions determined at 574 is then outputas indicated by arrow 596 to yield the wrist gimbal 699 joint positions.

Referring now to FIG. 27 of the drawings, the delta alignment operation558 will now be described in greater detail. The pincher formationorientation relative to the end of the articulated arm 712 is fed into asolution block 562 in similar fashion to the absolute alignmentoperation described with reference to FIG. 25. From 562, the segments ofinfinite solutions are forwarded to a selection block at 569. At 569,the position of arm member 702D at joint 704D for the solution which isclosest to the actual master joint position is selected. The position ofarm member 702D is then fed to an Inverse Kinematics block at 571together with the pincher formation orientation relative to the end ofthe articulated arm 712, as indicated by arrow 573, to yield the angularpositions of the wrist gimbal joints, as indicated by arrow 579. At 575,the joint positions indicated by arrow 579, the actual master jointposition, and information relating to what the current branch, namelyright pincher to right finger, or right pincher to left finger, arecompared to determine an appropriate angular position of the pincherformation 706 about axis 1, in similar fashion as described above at 590with reference to FIGS. 25 and 26. In this case, it is assumed that onlya minor misalignment has taken place so that the current master jointpositions and the current branch are used in place of the referencesolution 594 and the reference branch 592. The angular position of thepincher formation about axis 1, together with the other joint positionsdetermined at 571 is then output as indicated by 577 to yield the wristgimbal joint positions.

While the invention has been particularly shown and described withreference to preferred embodiments thereof, it will be understood bythose skilled in the art that various changes in form and details may bemade thereto without departing from the spirit and scope of theinvention as defined in the accompanying claims. Furthermore, wherereference is made to Cartesian coordinate systems, polar coordinatesystems, or cylindrical systems, or the like, can be used instead.

What is claimed is:
 1. A method of establishing a desired orientationalrelationship between a hand-held part of a master control and an endeffector of an associated slave as viewed in an image displayed on aviewer, the method including causing the end effector to remainstationary; determining a current orientation of the end effectorrelative to a viewing end of an image capturing device operativelyassociated with the viewer; determining a desired correspondingorientation of the hand-held part of the master control relative to theviewer at which the desired orientational relationship would beestablished; and moving the hand-held part of the master control intothe desired corresponding orientation.
 2. A method as claimed in claim1, which further includes determining a current position of the endeffector relative to the viewing end of the image capturing deviceoperatively associated with the viewer; and determining a desiredcorresponding position of the hand-held part of the master controlrelative to the viewer at which position a desired positionalrelationship between the hand-held part of the master control and theend effector would be established; and moving the hand-held part of themaster control into the desired corresponding position.
 3. A method asclaimed in claim 2, in which moving the hand-held part of the mastercontrol into the desired corresponding position includes manually movingthe hand-held part.
 4. A method as claimed in claim 1, in which movingthe hand-held part of the master control into the desired correspondingorientation includes supplying electrical current to electrical motorsassociated with the hand-held part thereby to cause the hand-held partto be moved into the desired orientation.
 5. A method as claimed inclaim 1, in which moving the hand-held part of the master control intothe desired corresponding orientation includes manually moving thehand-held part.
 6. A method of establishing a desired orientationalrelationship between a hand-held part of a master control and an endeffector of an associated slave as viewed in an image displayed on aviewer, the method including causing the end effector to remainstationary; determining a current orientation of the end effectorrelative to a viewing end of an image capturing device operativelyassociated with the viewer, by computing the orientation relative to areference coordinate frame positioned at the viewing end of the imagecapturing device; determining a desired corresponding orientation of thehand-held part of the master control relative to the viewer at which thedesired orientational relationship would be established; and moving thehand-held part of the master control into the desired correspondingorientation.
 7. A method as claimed in claim 6, which further includesdefining a coordinate frame on the end effector, the orientation of theend effector then being defined by the orientation of the coordinateframe on the end effector relative to the reference coordinate frame. 8.A method as claimed in claim 6, in which determining the desiredcorresponding orientation of the hand-held part of the master controlrelative to the viewer includes computing an orientation of thehand-held part relative to a reference coordinate frame generally at theviewer.
 9. A method as claimed in claim 8, in which computing theorientation of the hand-held part relative to the reference coordinateframe generally at the viewer includes defining a coordinate frame onthe hand-held part, the orientation of the hand-held part then beingdefined by the orientation of the coordinate frame on the hand-held partrelative to the reference coordinate frame at the viewer.
 10. A methodas claimed in claim 8, in which determining the desired correspondingorientation of the hand-held part includes determining an orientation ofthe hand-held part relative to the reference coordinate frame generallyat the viewer which is the same as the orientation of the end effectorrelative to the reference coordinate frame at the image capturingdevice.
 11. A method as claimed in claim 8, in which determining thedesired corresponding orientation of the hand-held part includesdetermining an orientation of the hand-held part relative to thereference coordinate frame generally at the viewer which is offsetrelative to the orientation of the end effector relative to thereference coordinate frame at the image capturing device.
 12. A methodof establishing a desired orientational relationship between a hand-heldpart of a master control and an end effector of an associated slave asviewed in an image displayed on a viewer, the method including causingthe end effector to remain stationary; determining a current orientationof the end effector relative to a viewing end of an image capturingdevice operatively associated with the viewer; determining a desiredcorresponding orientation of the hand-held part of the master controlrelative to the viewer at which the desired orientational relationshipwould be established, the master control having a plurality ofconfigurations which yield the desired corresponding orientation;computing a preferred configuration of the master control at which thehand-held part is at the desired corresponding orientation; and movingthe hand-held part of the master control into the desired correspondingorientation.
 13. A method as claimed in claim 12, in which computing thepreferred configuration includes examining a plurality of solutionscorresponding to at least one segment of solutions.
 14. A method asclaimed in claim 13, in which examining the plurality of solutions ofthe segment includes selecting an arbitrary solution and then followingan iterative procedure to compute the preferred solution.
 15. A methodas claimed in claim 14, in which following an iterative procedure todetermine the preferred solution includes using a cost function.
 16. Amethod as claimed in claim 15, in which the preferred solution relatesto the solution having a lowest cost function value.
 17. A method ofestablishing a desired orientational relationship between a hand-heldpart of a master control and an end effector of an associated slave asviewed in an image displayed on a viewer, the method including causingthe end effector to remain stationary; determining a current orientationof the end effector relative to a viewing end of an image capturingdevice operatively associated with the viewer; determining a currentposition of the end effector relative to the viewing end of the imagecapturing device operatively associated with the viewer by computing theposition relative to a reference coordinate frame positioned at theviewing end of the image capturing device; determining a desiredcorresponding orientation of the hand-held part of the master controlrelative to the viewer at which the desired orientational relationshipwould be established; determining a desired corresponding position ofthe hand-held part of the master control relative to the viewer at whichposition a desired positional relationship between the hand-held part ofthe master control and the end effector would be established; moving thehand-held part of the master control into the desired correspondingorientation; and moving the hand-held part of the master control intothe desired corresponding position.
 18. A method as claimed in claim 17,in which computing the position of the end effector relative to thereference coordinate frame at the viewing end of the image capturingdevice includes defining a coordinate frame on the end effector, theposition of the end effector then being defined by the position of thecoordinate frame on the end effector relative to the referencecoordinate frame.
 19. A method as claimed in claim 17, in whichdetermining the desired corresponding position of the hand-held part ofthe master control relative to the viewer includes computing a positionof the hand-held part relative to a reference coordinate frame generallyat the viewer.
 20. A method as claimed in claim 19, in which computingthe position of the hand-held part relative to the reference coordinateframe generally at the viewer includes defining a coordinate frame onthe hand-held part, the position of the hand-held part then beingdefined by the position of the coordinate frame on the hand-held partrelative to the reference coordinate frame at the viewer.
 21. A methodas claimed in claim 19, in which determining the desired correspondingposition of the hand-held part includes determining a position of thehand-held part relative to the reference coordinate frame generally atthe viewer which is the same as the position of the end effectorrelative to the reference coordinate frame at the image capturingdevice.
 22. A method as claimed in claim 19, in which determining thedesired corresponding position of the hand-held part includesdetermining a position of the hand-held part relative to the referencecoordinate frame generally at the viewer which is offset relative to theposition of the end effector relative to the reference coordinate frameat the image capturing device.
 23. A method of establishing a desiredorientational relationship between a hand-held part of a master controland an end effector of an associated slave as viewed in an imagedisplayed on a viewer, the method including causing the end effector toremain stationary; determining a current orientation of the end effectorrelative to a viewing end of an image capturing device operativelyassociated with the viewer; determining a desired correspondingorientation of the hand-held part of the master control relative to theviewer at which the desired orientational relationship would beestablished; moving the hand-held part of the master control into thedesired corresponding orientation, which includes manually moving thehand-held part; and causing the hand-held part to be locked in positionwhen its orientation is moved into an orientation in register with thedesired corresponding orientation.
 24. A method of establishing adesired orientational relationship between a hand-held part of a mastercontrol and an end effector of an associated slave as viewed in an imagedisplayed on a viewer, the method including causing the end effector toremain stationary; determining a current orientation of the end effectorrelative to a viewing end of an image capturing device operativelyassociated with the viewer; determining a desired correspondingorientation of the hand-held part of the master control relative to theviewer at which the desired orientational relationship would beestablished; moving the hand-held part of the master control into thedesired corresponding orientation, which includes manually moving thehand-held part; and providing a signal to an operator manually movingthe hand-held part when the hand-held part is moved into an orientationin register with the desired corresponding orientation.
 25. A method ofestablishing a desired orientational relationship between a hand-heldpart of a master control and an end effector of an associated slave asviewed in an image displayed on a viewer, the method including causingthe end effector to remain stationary; determining a current orientationof the end effector relative to a viewing end of an image capturingdevice operatively associated with the viewer; determining a currentposition of the end effector relative to the viewing end of the imagecapturing device operatively associated with the viewer; determining adesired corresponding orientation of the hand-held part of the mastercontrol relative to the viewer at which the desired orientationalrelationship would be established; determining a desired correspondingposition of the hand-held part of the master control relative to theviewer at which position a desired positional relationship between thehand-held part of the master control and the end effector would beestablished; moving the hand-held part of the master control into thedesired corresponding orientation; and moving the hand-held part of themaster control into the desired corresponding position by supplyingelectrical current to electrical motors associated with the hand-heldpart thereby to cause the hand-held part to be moved into the desiredposition.
 26. A method of establishing a desired orientationalrelationship between a hand-held part of a master control and an endeffector of an associated slave as viewed in an image displayed on aviewer, the method including causing the end effector to remainstationary; determining a current orientation of the end effectorrelative to a viewing end of an image capturing device operativelyassociated with the viewer; determining a current position of the endeffector relative to the viewing end of the image capturing deviceoperatively associated with the viewer; determining a desiredcorresponding orientation of the hand-held part of the master controlrelative to the viewer at which the desired orientational relationshipwould be established; determining a desired corresponding position ofthe hand-held part of the master control relative to the viewer at whichposition a desired positional relationship between the hand-held part ofthe master control and the end effector would be established; moving thehand-held part of the master control into the desired correspondingorientation; and moving the hand-held part of the master control intothe desired corresponding position, which includes manually moving thehand-held part and causing the hand-held part to be locked in positionwhen its position is moved into a position in register with the desiredcorresponding position.
 27. A method of establishing a desiredorientational relationship between a hand-held part of a master controland an end effector of an associated slave as viewed in an imagedisplayed on a viewer, the method including causing the end effector toremain stationary; determining a current orientation of the end effectorrelative to a viewing end of an image capturing device operativelyassociated with the viewer; determining a current position of the endeffector relative to the viewing end of the image capturing deviceoperatively associated with the viewer; determining a desiredcorresponding orientation of the hand-held part of the master controlrelative to the viewer at which the desired orientational relationshipwould be established; determining a desired corresponding position ofthe hand-held part of the master control relative to the viewer at whichposition a desired positional relationship between the hand-held part ofthe master control and the end effector would be established; moving thehand-held part of the master control into the desired correspondingorientation; and moving the hand-held part of the master control intothe desired corresponding position, which includes manually moving thehand-held part and providing a signal to an operator manually moving thehand-held part when the hand-held part is moved into a position inregister with the desired corresponding position.
 28. A method ofestablishing a desired orientational relationship between a hand-heldpart of a master control and a surgical end effector of an associatedsurgical instrument slave in a telesurgical apparatus, the methodincluding causing the end effector to remain stationary; determining acurrent orientation of the end effector relative to a viewing end of animage capturing device operatively associated with a viewer; determininga desired corresponding orientation of the hand-held part of the mastercontrol relative to the viewer at which the desired orientationalrelationship would be established; and moving the hand-held part of themaster control into the desired corresponding orientation by supplyingelectrical current to electrical motors associated with the hand-heldpart thereby to cause the hand-held part to be moved into the desiredorientation.
 29. A method as claimed in claim 28, in which the initialdetermining step includes computing the orientation relative to areference coordinate frame positioned at the viewing end of the imagecapturing device.
 30. A method as claimed in claim 29, which furtherincludes defining a coordinate frame on the end effector, theorientation of the end effector then being defined by the orientation ofthe coordinate frame on the end effector relative to the referencecoordinate frame.
 31. A method as claimed in claim 29, in whichdetermining the desired corresponding orientation of the hand-held partof the master control relative to the viewer includes computing anorientation of the hand-held part relative to a reference coordinateframe generally at the viewer.
 32. A method as claimed in claim 31, inwhich computing the orientation of the hand-held part relative to thereference coordinate frame generally at the viewer includes defining acoordinate frame on the hand-held part, the orientation of the hand-heldpart then being defined by the orientation of the coordinate frame onthe hand-held part relative to the reference coordinate frame at theviewer.
 33. A method as claimed in claim 31, in which determining thedesired corresponding orientation of the hand-held part includesdetermining an orientation of the hand-held part relative to thereference coordinate frame generally at the viewer which is the same asthe orientation of the end effector relative to the reference coordinateframe at the image capturing device.
 34. A method as claimed in claim31, in which determining the desired corresponding orientation of thehand-held part includes determining an orientation of the hand-held partrelative to the reference coordinate frame generally at the viewer whichis offset relative to the orientation of the end effector relative tothe reference coordinate frame at the image capturing device.
 35. Amethod as claimed in claim 28, in which the master control has aplurality of configurations which yield the desired correspondingorientation, the method including computing a preferred configuration ofthe master control at which the hand-held part is at the desiredcorresponding orientation.
 36. A method as claimed in claim 35, in whichcomputing the preferred configuration includes examining a plurality ofsolutions corresponding to at least one segment of solutions.
 37. Amethod as claimed in claim 36, in which examining the plurality ofsolutions of the segment includes selecting an arbitrary solution andthen following an iterative procedure to compute the preferred solution.38. A method as claimed in claim 37, in which following an iterativeprocedure to determine the preferred solution includes using a costfunction.
 39. A method as claimed in claim 38, in which the preferredsolution relates to the solution having a lowest cost function value.40. A method as claimed in claim 28, which further includes determininga current position of the end effector relative to the viewing end ofthe image capturing device operatively associated with the viewer; anddetermining a desired corresponding position of the hand-held part ofthe master control relative to the viewer at which position a desiredpositional relationship between the hand-held part of the master controland the end effector would be established; and moving the hand-held partof the master control into the desired corresponding position.
 41. Amethod as claimed in claim 40, in which determining the current positionof the end effector relative to the viewing end of the image capturingdevice includes computing the position relative to a referencecoordinate frame positioned at the viewing end of the image capturingdevice.
 42. A method as claimed in claim 41, in which computing theposition of the end effector relative to the reference coordinate frameat the viewing end of the image capturing device includes defining acoordinate frame on the end effector, the position of the end effectorthen being defined by the position of the coordinate frame on the endeffector relative to the reference coordinate frame.
 43. A method asclaimed in claim 41, in which determining the desired correspondingposition of the hand-held part of the master control relative to theviewer includes computing a position of the hand-held part relative to areference coordinate frame generally at the viewer.
 44. A method asclaimed in claim 43, in which computing the position of the hand-heldpart relative to the reference coordinate frame generally at the viewerincludes defining a coordinate frame on the hand-held part, the positionof the hand-held part then being defined by the position of thecoordinate frame on the hand-held part relative to the referencecoordinate frame at the viewer.
 45. A method as claimed in claim 43, inwhich determining the desired corresponding position of the hand-heldpart includes determining a position of the hand-held part relative tothe reference coordinate frame generally at the viewer which is the sameas the position of the end effector relative to the reference coordinateframe at the image capturing device.
 46. A method as claimed in claim43, in which determining the desired corresponding position of thehand-held part includes determining a position of the hand-held partrelative to the reference coordinate frame generally at the viewer whichis offset relative to the position of the end effector relative to thereference coordinate frame at the image capturing device.
 47. A methodas claimed in claim 40, in which moving the hand-held part of the mastercontrol into the desired corresponding position includes supplyingelectrical current to electrical motors associated with the hand-heldpart thereby to cause the hand-held part to be moved into the desiredposition.
 48. A method as claimed in claim 40, in which moving thehand-held part of the master control into the desired correspondingposition includes manually moving the hand-held part.
 49. A method asclaimed in claim 48, which further includes causing the hand-held partto be locked in position when its position is moved into a position inregister with the desired corresponding position.
 50. A method asclaimed in claim 48, which further includes providing a signal to anoperator manually moving the hand-held part when the hand-held part ismoved into a position in register with the desired correspondingposition.
 51. A method of establishing a desired orientationalrelationship between a hand-held part of a master control and a surgicalend effector of an associated surgical instrument slave in atelesurgical apparatus, the method including causing the end effector toremain stationary; determining a current orientation of the end effectorrelative to a viewing end of an image capturing device operativelyassociated with a viewer; determining a desired correspondingorientation of the hand-held part of the master control relative to theviewer at which the desired orientational relationship would beestablished; and moving the hand-held part of the master control intothe desired corresponding orientation by manually moving the hand-heldpart.
 52. A method as claimed in claim 51, which further includescausing the hand-held part to be locked in position when its orientationis moved into an orientation in register with the desired correspondingorientation.
 53. A method as claimed in claim 51, which further includesproviding a signal to an operator manually moving the hand-held partwhen the hand-held part is moved into an orientation in register withthe desired corresponding orientation.
 54. A method of selecting apreferred angular position of a hand-held part of a master control abouta roll axis thereof, which corresponds with a current angular positionof an end effector about a roll axis thereof, where the hand-held partcan have a plurality of angular positions spaced at 180° intervalsrelative to each other which correspond with the current angularposition of the end effector, the method including determining theangular position of the end effector relative to an angular displacementrange about its roll axis; comparing the angular position of the endeffector relative to its angular displacement range with the angularpositions of the hand-held part, which correspond with the angularposition of the end effector, relative to an angular displacement rangeof the hand-held part; and selecting the angular position of thehand-held part relative to its angular displacement range which bestmirrors the angular position of the end effector relative to its angulardisplacement range.
 55. A method as claimed in claim 54, in which thehand-held part is orientationally adjustably mounted on an associatedsupport, the end effector is orientationally adjustably mounted on anassociated support, the method including the prior step of determining adesired orientation of the hand-held part which establishes a desiredorientational relationship between the hand-held part and the endeffector, the desired orientation of the hand-held part defining one ofthe corresponding angular positions of the hand-held part whichcorresponds to the current angular position of the end effector.
 56. Amethod as claimed in claim 55, in which the comparing step includesimposing a simulated orientational change on the hand-held part whichcorresponds to a simulated orientational change of the end effector inmoving its orientation into a predetermined reference orientation.
 57. Amethod as claimed in claim 56, which includes determining a referencesolution set of angular positions at 180° intervals relative to eachother for the simulated orientation of the hand-held part whichcorrespond to the angular position of the end effector at its simulatedpredetermined reference orientation.
 58. A method as claimed in claim57, which includes comparing the current end effector angular positionrelative to its angular displacement range with the angular positions ofthe reference solution set relative to the angular displacement range ofthe hand-held part.
 59. A method as claimed in claim 58, which includesselecting the angular position of the reference solution set relative tothe angular displacement range of the hand-held part which best mirrorsthe current angular position of the end effector relative to its angulardisplacement range.
 60. A method as claimed in claim 59, which includespredefining a solution branch of 360° intervals and predefining anothersolution branch of 360° intervals offset from the first solution branchby 180°, the selected angular position defining one of the solutionbranches.
 61. A method as claimed in claim 60, which includes comparingthe defined solution branch and the selected angular position of thereference solution set with the angular positions of the hand-held part,which correspond with the current angular position of the end effector,relative to the angular displacement range of the hand-held part andselecting that angular position relative to the angular displacementrange of the hand-held part which corresponds to the defined solutionbranch and which best mirrors the selected angular position of thereference solution set.
 62. A surgical system comprising: a mastercontroller having a hand-held part and an actuator; an end effector formanipulation of tissue; a display showing an image of the end effector;and a processor coupling the end effector to the master controller, theprocessor transmitting actuator signals to the actuator in response toan initial orientation of the hand-held part, the actuator moving thehand-held part from the initial orientation to a desired orientation inresponse to the actuator signals so that the hand-held part establishesa predetermined orientational relationship with the image of the endeffector.
 63. A surgical system as claimed in claim 62, wherein theprocessor determines a current orientation of the end effector relativeto a viewing end of an image capturing device operatively associatedwith a viewer; and determines the desired corresponding orientation ofthe hand-held part of the master controller relative to the viewer atwhich the desired orientational relationship would be established.
 64. Asurgical system as claimed in claim 62, wherein the processor determinesa current position of the end effector relative to a viewing end of animage capturing device operatively associated with a viewer; determinesa desired corresponding position of the hand-held part of the mastercontroller relative to the viewer at which position a desired positionalrelationship between the hand-held part of the master controller and theend effector would be established; and transmits actuation signals tothe actuator in response to an initial position of the hand-held part,the actuator moving the hand-held part from the initial position to adesired position in response to the actuator signals.
 65. A surgicalsystem as claimed in claim 62, wherein the processor determines anangular position of the end effector relative to an angular displacementrange about a roll axis thereof; compares the angular position of theend effector relative to its angular displacement range with angularpositions of the hand-held part of a master controller about a roll axisthereof, which correspond with the angular position of the end effector,relative to an angular displacement range of the hand-held part; andselects the angular position of the hand-held part relative to itsangular displacement range which best mirrors the angular position ofthe end effector relative to its angular displacement range.
 66. Amethod of establishing a desired orientational relationship between ahand-held part of a master control and a surgical end effector of anassociated surgical instrument slave in a telesurgical apparatus, themethod including causing the end effector to remain stationary;determining a current orientation of the end effector relative to aviewing end of an image capturing device operatively associated with aviewer; determining a desired corresponding orientation of the hand-heldpart of the master control relative to the viewer at which the desiredorientational relationship would be established; and moving thehand-held part of the master control into the desired correspondingorientation by supplying electrical current to electrical motorsassociated with the hand-held part.